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FOREWORD  FROM  THE  CONFERENCE  CHAIRMAN 


The  need  for  effective  weapon  system  training  is  receiving  increased  attention 
throughout  the  DoD  community  as  the  benefits  of  improved  student  performance  and 
reductions  in  training  costs  are  documented.  The  use  of  simulation  in  flight 
training  is  now  accepted  as  being  highly  effective  while  affording  significant 
reductions  in  training  costs  .  Similar  applications  of  training  and  simu*. *-ion 
technology  are  gaining  acceptance  in  such  diverse  areas  as  team  tjctics  training, 
surface  and  sub-surface  ASW,  conduct-of -f i re  systems,  and  air  cushion  v^hicl.e 
operation. 

Technological  advances  in  solid  state  devices,  design  archi tec tures ,  software, 
image  generation  and  display,  and  performance  measurement,  to  name  but  a  few,  are 
making  possible  increased  training  device  functionality,  effectiveness,  reliability, 
and  supportaDi li ty .  The  present  effectiveness  is  yielding  the  long  needed  docu¬ 
mentation  of  training  device  payoffs.  The  recent  Defense  Science  Board  study  on 
training  illustrates  recognition  of  the  growing  importance  of  cost-effective 
training  to  this  nation's  defense. 


Recent  world  events  have  demonstrated  the  impact  of  highly  trained  military  personnel 
on  national  defense.  World  attention  in  1982  has  been  focused  on  the  decisive 
role  training  has  played  at  sea,  in  the  air,  and  on  land.  These  experiences  will 
undoubtedly  affect  future  DoD  plans  and  lend  strong  support  for  continuation  of  the 
e mpha s n  training. 


Close  communication  between  the  military  and  industry  is  critical  to  advancements 
in  training. \  Without  a  clear  understanding  of  the  military  needs  and  problems  in 
training,  effective  solutions  are  doubtful.  Conversely,  military  planners  cannot 
capitalize  on  vhhe  technical  and  management  advances  of  industry  unless  they  are 
communicated .“-^he  goal  of  the  Interservice/ Indus  try  Training  Equipment  Conference 
is  to  serve  as  a  forum  to  foster  communication .  Through  technical  papers, 

panels,  equipment  exhibits,  and  individual  discussions,  problems  are  aired,  needs 
are  defined,  and  plans  are  specified  to  develop  al ternatives- and,  solutions. 

The  purpose  of  this  conference  is  communication  and^ihe  papers  contained  in  these 
proceedings  are  an  attempt  to  document  both  industry-ts  and  government's  views. 
These  papers  were  selected  to  present  a  wide  range  of  government  and  industry 
techniques,  issues,  problems,  and  needs. 


While  past  conferences  have  covered  a  wide  range  of  training  topics,  it  has 
become  increasingly  recognized  that  the  voice  of  the  user  must  be  heard,,  It  is  not 
eno'ugTT'To' bear ---from  the  acquisition,  requirements,  design  or  support  specialists  . 
'fc’he  training  equipment  user  is  da; ./  confronted  with  a  diversity  of  unique 
difficulties  which  are  frequently  unseen  by  others  in  the  training_ cycle ^  Complica¬ 
ting  the  problem  is  that  few  vehicles  are  available  to  the  user  to  voicejthese 
issues.  jrThe  Four  th  '_Jin  terser  vice /Indus  try  Training  Equipment- CbltTerence'  is, 
therefore,  focused  on  the  user.  Our  goal  is  to  communicate  the  user's  views  to 
those  in  government  and  industry  and  hopefully  improve  the  resultant  training 
product . 

\  Dr.  James  A.  Gardner 

Conference  Chairman 


s  f"  7 

J 

■:CT£% 


,  5  0  1932  • 

t) 


A 


This  document  has  been  approved 
for  public  release  and  sale;  its 
istribution  is  unlimited. 


INTERSERVICE  EXECUTIVE  COMMITTEE 


INTERSERV  ECE  STEERING  COMMITTEE 


Chairman:  CAPT  J.T.  McHugh,  USN , 

NAVTRAEQUIPCEN 

COL  A . J .  Castellana,  USMC 
Marine  Corps  Liaison  Officer 

COL  T.W.  Honeywil 1 ,  USAF 
Deputy  for  Simulators,  ASD 

COL  D.M.  Campbell,  USA 
PM  Trade 

NSIA  LIAISON 

COL  P.  J.  Cole,  USA  Ret) 
Director,  National  Activities 

Robert  W.  Layne,  Chairman 
Trainers  &  Simulators 


Chairman:  Thomas  W.  McNaney,  NAVTRAEQUIPCEN 

Ralph  G.  Nelson,  PM  Trade 

LT  COL  R.E.  Fairfield,  USMC 
NAVTRAEQUIPCEN 

Robert  R.  Swab,  AsD 


CONFERENCE  COMMITTEE 

Chairman:  Dr.  James  Gardner,  Honeywell,  Inc. 

COMMUNICATION  COMMITTEE 

Chairman:  Victor  Faconti,  Singer  Link  Flight  Simulation  Division 


EXHIBIT  COMMITTEE 


Chairman:  Martin  Morganlander ,  Gould  Simulation  Systems 

Members:  Toby  Royal,  Gould  Simulation  Systems 

James  Caspar,  McDonnell  Douglas  Astronautical  Company 
William  K.  Lee,  McDonnell  Douglas  Astronautical  Company 
John  M.  Gifford,  Sperry  System  Managements 


FACILITIES  COMMITTEE 


Chairman:  Bob  Witsiil,  General  Electric,  Dayton  Beach,  FL 

Members:  Ken  Kilner,  General  Electric 

Harley  E.  Hoffman,  General  Electric 
James  Bishop,  NAVTRAEQUIPCEN 


PUBLICITY  COMMITTEE 


Chairman:  Janes  Zullo,  Applied  Devices  Corp.,  Kissimmee,  FL 

Members:  Allen  Q.  Collier,  NAVTRAEQUIPCEN 


REGISTRATION  COMMITTEE 


Chairman : 


John  M.  Gifford,  Sperry  Systems  Management,  Orlando,  FL 


PROGRAM  COMMITTEE 


Chairman:  Ralph  T.  Davis,  Jr.,  Cubic  Corporation 

Special  Assistant:  A.  Will'u"'  Herzog 

TECHNICAL  SUBCOMMITTEE 


Chairman:  Rodney  3.  Rougelot,  Evans  &  Sutherland 

Special  Assistant:  Robert  D.  Stirland 

Members:  Dr.  David  Abbott,  University  of  Central  Florida 

Robert  W.  Beck,  ASD  Wright  Patterson  AFB 
James  E.  Bishop,  NAVTRAEQUIPCEN 

Dallas  Butler,  Gould,  Simulation  Systems  Division 
Arthur  B.  Doty,  ASD  Wright  Patterson  AFB 
Muraock  McKinnon,  CAE  Electronics,  Ltd. 

Dr.  Robert  Fuege,  Mathetics,  Inc. 

Lawrence  Gallagher,  Xerox  Electro-Optics  Systems 

G.  Larry  Graham,  Grumman  Aerospace 

Harold  Kottman,  ASD  Wright  Patterson  AFB 

Dr.  Norman  R.  Potter,  Systems  Research  Labs,  Inc. 

Dr.  Renata  Schmidt,  Consultant 

Dr.  Thomas  Sitterley,  Boeing  Aerospace 

Joe  Thompson,  PM  TRADE 

Steve  J.  Trencansky,  Singer  Link  Division 
Robert  Valone,  NAVAIRSYSCOM 
Dr.  Richard  Walker,  Coursware 

MANAGEMENT  SUBCOMMITTEE 

Chairman:  Joseph  Andreani,  Reflectone  Corporation 

Members:  Cyril  Brayne,  Industry  Trade  4  Commerce,  Canada 

C.  J.  Chappell,  III,  NAVTRAEQUIFCEN 
Robert  E.  Coward,  ASD,  Wright  Patterson  AFB 
David  Greene,  McDonnell  Douglas  Corp. 

John  W .  Hammond,  AAI  Corporation 
Karl  Jackson,  PM  VhADE 

Russell  Johnson,  ASD  Wright  Patterson  AF3 
Eugene  McGinnis,  SAI  Corporation 
George  E.  Paler,  Goodyear  Aerospace 
Stanley  Plass,  Hughes  Aircraft  Co. 

Jon  Schreiber,  NAVAIRSYSCOM 
William  Simcox,  Cubic  Corporation 


USER  SUBCOMMITTEE 

Chairman:  Jack  C.  Bockus,  Odgen/ALC 

Members:  CDR  M.  Carpenter,  USN,  NAS  Memphis 

MAJ  J.  Burch,  LJSAF,  Scott  AFB 

COL  M.D.  Calnan ,  CAF,  National  Defense  Haqtrs. 

Richard  Dyer,  ATSC,  Ft.  Eustis,  VA 

John  Landers,  NAVAIRSYSCOM 

LTC  R.  R.  Rogers,  USAF,  Offutt  AFB 

Wayne  Wheatley,  NAVTRAEQUIPCEN,  Regional  Office, 

Dr.  Ruth  rfienclaw,  Honeywell,  Inc. 

Sam  Worrell,  Analysis  4  Technology,  Inc. 

SPEAKER  SUBCOMMITTEE 

Chairman:  RADM  0.  H.  Oberg,  USN (Ret),  Cubic  Corporation 
Members:  LT .  D.L.  Norman,  NAVTRAEQUIPCEN 


Atlantic 


TABLE  OF  CX3MTEOTS 


PAGE 

SESSL'M  T-B  -  Management  Considerations  #1 
Chairman:  Robert  E.  Coward 

Role  of  the  Joint  Logistics  Commanders  Joint  Technical  Coordinating  Group 
on  Simulators  Training  Devices  (JTCG-STD) 

LTCOL  G.R.  Winters  II,  USAF,  Dr.  Ronald  Hoger,  Jon  Schreiber  and 

Willard  D.  Haugen  .  i 

Getting  User  Requirements  into  the  Development-to-Delivery  Loop 

Frederick  W.  Snyder  . .  7 

SESSION  I-C  -  Visual  Technology  #1 
Ch airman:  Robe r t  Bee k 

Enhancing  the  Computer  Generated  Illusion 

Neal  L.  Mayer  and  Michael  A.  Cosman  .  13 

Simplified  Scene  Modeling  Using  Curved  Surfaces  and  Texturing 

Dr.  Geoffrey  Y.  Gardner  and  Bob  M.  Gel  man  .  23 

Exploding  Techniques  for  Computer  Image  Generation  Objects 

John  L.  Booker,  Charles  Csuri,  Michael  Collery  and  David  Zeltzer  .  31 

Computer  Synthesized  Imagery  -  A  New  Way  to  Realistic  Visual  Simulation 

Dr.  Robert  Stickel  .  35 

SESSION  I-D  -  Training  Systems  #1 
Chairman:  Paul  W£tmpner 

Ground  Forces  Training  Devices  &  Techniques:  Warsaw  Pact  Countries 

Helen  S.  Harrington  .  41 

MACE/MICRO-DISC:  A  Microcomputer-Videodisc  Battle  Simulation  System 

Dr.  Lawrence  T.  Brekka  and  Louis  Charity  . .  47 

Overview  of  an  Onboard  Training  Device 

Joseph  Ricci,  Jr .  53 

Thermal  Signature  Targets  for  Gunnery  Training 

Dr.  Robert  T.  Dybas  .  59 

SESSION  II-A  -  Defining  Needs  of  Users 
Chairman:  Barry  C.  Holt 

The  View  From  the  Other  End  of  the  Microscope  or  I ‘d  Rather  be  Flying 

LTCOL  Stephen  R.  Olsen,  USAF  .  75 

Tbe  User's  Role  in  Major  Training  System  Acquisitions  -  as  Perceived  by  the 
Developer 

George  C.  Barcus  .  79 

Save  our  Simulators  (SOS):  A  Distress  Call  From  an  Operational  User 

CDR  Kevin  M.  Smith,  USNR  . ; .  83 

Identifying  Necessary  Goals  &  Objectives  for  Training  Systems:  A  Needs 
Assessment  Approach 

Dr.  Charles  A.  Beagles  and  Dr.  Dee  H.  Andrews  . . .  9^ 


EEST  AVAILABLE  COPY 


PAGE 

SESSION  I  I-D  -  Management  Considerations  #2 
Chairman:  Robert  E.  Coward 

'Itie  Cost- -If  feet: iveness  of  Military  Training 

Jesse  Oriansky,  Joseph  String  and  CAPT  P.R.  Chate^itr,  USN  . .  97 

SESSION  I T-C  -  Trainer  Cost  Ef foot i veness 
Chai rman :  Jim  Hi shop 

PAM  arxi  Support  Considerations  in  Army  Training  Device  Development 

Francis  King,  Philip  E.  Sprinkle  and  Keith  M.  Gardner  .  Ill 

Low  Cost  Aircrew  Training  Systems 

Dr.  Arthur  S.  Plaiwes  .  119 

Tactical  Ground  Attack:  On  the  Transfer  of  Training  From  Flight  Simulator  to 
Operational  RED  FLAG  Range  Exercise 

Ronald  Hughes,  Rebecca  Brooks,  Douglas  Graham,  Ray  Sheen  and  Tom  Dickens  .  127 

Evaluation  of  the  Army  Maintenance  Training  find  Evaluation  Simulation 
Systems  (AtfTESS) 

Dr.  Robert.  A.  Evans  and  Dr.  Angelo  Mirabella  . . . . .  131 

SESSION  I I-D  -  Instructional  Technology 
Chairman:  Dr.  Loon  H.  Nawrocki 

ARI's  Research  Program  to  Determine  Training  Simulator  Characteristics 

Dr.  Robert  T.  Hays  .  137 

Implementation  of  Instructional  Features  on  Maintenance  Trainers 

Larry  Rude  and  SMSqt  William  Lippke  . . .  143 

Automated  Performance  Measurement:  An  Overview  and  Assessment 

Joseph  L.  Dickmar.  .  153 

Tactical  Training  Methodologies  for  Ground  Forces  Command  Center  and  Their 
Impl  f: mien  t  a  t  i  on 

Arie  Leider  .  167 

SESSION  III-P  -  Software  Management  Issues 
Chairman:  Robert  Schwing 

Use  of  Word  Processing  in  Data  Development  and  Review 

Frederick  F.  Beck  and  Richard  W.  Woolsey  .  169 

Baselined  Software  Configuration  Management:  An  Automated  Approach 

Mark  M.  Hargrove  .  179 

Using  Software  Development  Facilities  to  Improve  Software  Qjality 

Steve  J.  Trencansky,  Dennis  W.  Meehl  and  Hugh  C.  Romine  .  189 

SESSION  III-C  -  Visual  Technology  #2 
Chairman:  Jim  Burns 

360  Degrees  Visual  Target  Simulation  for  Outdoor  Field  Training 

Sean  Amour  and  Joseph  A.  LaRussa  .  195 

The  Trend  Tbwards  Area  of  Interest  in  Visual  Simulation  Technology 

Dr.  A.  Michael  Spooner  . . . . . . .  205 

The  Technical  Contributions  of  the  Tactical  Combat  Trainer  Development  Program 

James  D.  Basinger,  John  M.  Wilson  and  Robert  A.  Fisher  .  217 

Data  Base  Generation  System  for  Computer  Generator  Images  and  Digital 
Radar  Landmass  Simulation  Systems 

Lt.Col  Manfred  Haas,  Diether  Elflein  and  Peter  Gueldenpf ennis  .  231 

u  BEST  AVAILABLE  COPY 


PAGE 


SESSION  III -D  -  Maintenance  Training 
Chairman :  G.  Larry  Graham 


Ar  .iysis  of  Fidelity  Requirements  for  Simulated  Electronic  Maintenance  Training 
Equipment 

Dr.  L.  Bruce  McDonald,  Grace  Waldrop  and  Dr.  Richard  E.  Reynolds  .  237 

The  Integration  of  Videodisc,  CAI  and  3D  Simulation  for  Skills  Training 

James  R.  Stonge  . .  24  3 

A  Computer-Based  Job-Aid  for  Maintaining  Complex  Military  Hardware 

David  E.  Stone,  David  Mudrick  and  Lois  Wilson  . . .  249 

Lessons  Learned  in  the  Application  of  Simulation  to  Jet  Engine  Maintenance  Training 
Thomas  J.  Stillings,  Dr.  James  D.  Riley  and  MSGT  M.T.  Wagner  .  255 


SESSION  IV-C  -  Training  Equipment  Technology 
Chairman:  Richard  Jarvis 

The  Feasibility  of  Employing  an  In-cockpit  Device  to  Provide  Motion  Cues  to  the  Pilot 
of  a  Flight  Simulator 

Frank  M.  Cardullo  and  Steve  K.  Butrimas  . 

Prototype  Specifications  to  Support  High  Resolution-Sensor  Simulators 

Ronald  J.  Pierce  . 


Simulation  Methods  for  High  Resolution  Analysis  Sonars 

J.A.H.  Shaw  and  T.J.  Hunter  . 

Range-Dependent  Ocean  Acoustic  Transmission  Loss  Calculations  in  a  Real-time 
Framework 

John  F.  Miller  . 


SESSION  IV-D  -  Front  End  Analysis 
Chairwoman:  Dr  Renata  Schmidt 

The  Integration  of  Models  Used  for  Training  Equipment  Requirements  Deprivation: 
The  Instructional  Systems  Development,  Research  Design,  and  Systems  Engineering 
Processes 

Annette  Gnagy  . 

Application  of  Instructional  System  Development  Techniques  to  Team  Training 
Dr.  Frederick  M.  Ball  . 

Collective  Front-end  Analysis:  A  Mission-Based  Approach 

Dry an  E.  Brett  . 


5F3SI0N  V-B  -  Training  Systems 
Chairman:  John  Hammond 

Undersea  Warfare  Training  &  Readiness  Assessment.  System  Requirements 
Bradford  A.  Becken  . 

Onboard  Training  Program  Development:  lessons  Learned 

David  A.  Sc'noiwe  . . 


User  Guidelines  for  Deck  Officer  Training  Systems 

Dr.  Thomas  J.  H„mmell,  John  W.  Gvnther  and  Joseph  J.  Puglisi  .  333 

Computer  Aided  ..  stn. :t ion/Computer  Managed  Instruction:  How  Much  is  Enough? 

Philip  J.  Fis  ^  at:d  ■  arry  H.  Nowell  .  34  7 


iii 


BEST  AVAILABLE  COPY 


PAGE 


SESSION*  V-C  -  Computer  Systems 
Cha i mum  :  Phi l ip  S .  Babe 1 

Computer  Program  Documentation  -  What  is  Overkill? 

Karen  R.  Bailsman  .  357 

Application  of  Microcomputers  to  the  Simualtor  "Linkage"  Problem 

Sieve  Sc* idens ticker  . . .  353 

Evaluation  of  Computer  Configurations  for  Simulation 

Leonard  D.  Healy,  Bruce  Baker  and  Gerald  A.  Wyndle  .  3C9 


SESSION  V-D  -  Performance  Ev.aluat.ion 
Chairman:  Dr.  Nontuiu  P .  Potter 

Assessment  of  Simulator  Visual  Cueing  Effectivenss  by  Psychophysical  Techniques 
Joe  DeMaio  and  Rebecca  Bi  00k s  . . 


Methodology  to  Assess  In-flight.  Performance  For  Air-to-Air  Combat  Training 

Anthony  P .  Ciavarell  i ,  Jr . 7. . .  .*77 .  383 

Effectiveness  Evaluation  for  Air  Combat  Training 

Dr.  James  McGuinness,  John  H.  Bouwnan  and  Joseph  A.  Puig  .  391 

Simulation  and  Training  for  Aircraft  Carrier  Landings 

Daniel  P.  Westra  . .  397 


SESSION  VI -A  -  Training  Effectiveness  and  Instructor  Interface 
Chairman:  COL  Brure  E.  Ross,  USAF 

A  Model  for  Determining  Cost  and  Training  Ef fectiveness  Tradeoffs  for  Training 


Equipment 

Dr.  Ruth  A.  Wienclaw  and  COL  Frank  E.  Hines  .  405 

Training  Effectiveness  Evaluation  of  DeviceA/F37A-T59 

CAP!’  J.  Kotor  a,  USAF  and  CAPT  W.F.  Siebert,  USAF  .  417 

The  Tnstructor/Operator  Station:  Design  /or  the  User 

William  T.  Harris,  Walter  M.  Komanski  and  Dr.  William  M.  Hinton  .  427 


SESSION  VI-R  -  Acquisition 
Cha irmnu:  Stan  PI ass 

Needed:  A  State-of-the-Art  Integrated  Logistic  Support  Acquisition  Strategy 


Maurice  Wi.nsor  and  Thomas  E.  CXcise  .  437 

Some  Analytical  Thoughts  on  Oie  Answer  to  the  Services  Training  Manpower 
Crunch  -  Training  by  Contractors 

Dr.  Jack  Ballard  .  443 

The  Procurement  Communication  Game 

Dr.  Mona  Crissey  and  Stella  Sherman  . .  447 

A  DSMC  Simulation:  Decision  Exercises 

George  R.  McAleer,  Jr . .  453 


SESSION  VI-C  -  Training  Systems  #2 
Chairman:  Dr.  Thomas  Sitterley 

an  Interactive,  High  Threat  Combat  Environment 
CAPT  R.L.  Sheen  and  Thomas  M.  Dickens  .  459 


REDFLAG  Simulation:  Development  of 
Clarence  Stephens,  Patricia  Widder, 


1LABLE  COPY 


lv 


PAGE 


SESSION  Vr-C  -  Training  Syst: oma  #?  ( Con t. •  d ) 

TRIAD  -  An  Approach  to  Embedded  Simulation 

Peter  Cross  and  Andy  Olson  . . . 

F-IG  &  A- .10 A  OPT  Simulators  -  Flight  Systems  Development  and  Test 


K.L.  Johnson  and  M.P.  Bogumill  .  48b 

Emphasis  on  Reliability,  Maintainability  &  Quality  Assurance 

CAPT  J.R.  Seeley  .  493 


SESSION  VI I- A  -  User  Issues 
Chairman:  W.G.  Galloway 


The  Military  Airlift  Command  Aircrew  Training  Devices  Program 

MAJ  John  Orier  and  MAJ  Joe  Burch  .  4  95 

VTXTS  -  A  Cooperative  Effort  Between  the  User,  the  Chief  of  Naval  Air  Training,  and 
the  Contractor,  McDonnell  Douglas  Corporation 

Jerome  J.  Schi'-k,  CDR  W.D.  Jones  and  Eldon  W.  Riley  .  501 

Training  Device  Support  Concepts  for  the  Future:  A  Problem  Solving  Approach  to 
Cost  Reduction 

Robert  E.  LeVan,  Robert  A.  Cross  and  Harry  E.  Ros coe  .  507 

NBC  Training  Requirements  for  the  Integrated  Battlefield 

Gary  E.  Harvey  .  515 


SESSION  VTI-C  -  Visual  Technology  #3 
Chairman:  Murdock  McKinnon 


Characteristics  of  Flight  Simulator  Visual  Systems 

Briar.  Welch  . ' .  523 

Filtering  Simulated  Visual  Scenes  -  Spatial  and  Temporal  Effects 

W.  Marvin  Bunker  . . .  531 

Texture  in  a  Low  Cost  Visual  System 

Philip  T.  Skolmoski  and  Michael  Fortin  .  541 

Computer  Genera ted/Synthesized  Imagery  (CGSI) 

Carl  Graf  and  Dorothy  M.  Baldwin  . .  .  549 


BEST  AVAILABLE  COPY 


V 


AD  POO 01  5  4 


ROM  OF  THE  JOtNT  LOCISTICS  COMMANDERS  JOINT  TECHNICAL  COORDINATING  CROW 
ON  SIMULATORS  AND  TRAINING  DEVITOS  (JTCG-STD) 

Lieutenant  Colonel  George  R.  Winters,  II,  USAF 
Headquarter's  Air  Force  Systems  Command 

Doctor  Ronald  Hofei'' 

Project  Manager  for  Training  Devices,  U.S.  Army 

Jon  Schreiber 
Naval  Air  Systems  Command 

-  ‘  \  Willard  D.  Haugen 

°gden  Air  Logistics  Center,  Air  Force  Logistics  Command 

v  j  ABSTRACT  - - 

v.ir  ,  ^  1°^.  Command ersMJLC)  are  composed  of.  the  commanders  of  the  US  Army 

Materiel  Readiness  and  Development  Command,  Air  Force  Logistics  Command,  and  Air  Force 
Systems  Command  as  well  as  the  Chief  of  *aval  Material.  The  JLC  meet  regularly  with 
the  Deputy  Secretary  of  Defense.  Their^purpose  is  to  resolve  common  concerns  and, 
where  Possible,  to  achieve  efficiencies  by  combining  efforts.  The  day-to-day  activities 
of  theJfLC  are  carried  out  by  joint  panels  and  groups. 
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such  group  is  the  Joint  Technical  Coordinating  Group  on  Simulators  and  Training 
CJTCG  STD).  Its  purpose/  as  stated  in  its  charter  from  the  JLC/^is  to  identify 
ities  to  coordinate  or  consolidate  programs  in  research  and  development. 

Ion,  and  operation  nrd  support  of  training  devices  and  to  implement  plans  to 
he  cost  and/or  increase  the  effectiveness  of  military  simulators  and  training 


t  is  a  large  order.  To  accomplish  it  requires  breaking  it  into  manageable 
Each  year  the  JTCG-STD  will  review  proposals  for  efforts  which  have  High 
to  two  or  more  services.  The  best  of  these  will  be  chosen  to  h«  presented  to 
as  candidates  for  JLC  sponsorship.  JLC  sponsored  tasks  will  be  managed  by  a 
ocunand  with  assistance  from  the  others. 


The  first  three  tasks  chosen  for  JLC  sponsorship  are  to  develop  (1)  a  standard 
Defense  Mapping  Agency  data  base  transfoi mat  ion  program,  (2)  a  library  of  standard 
electronic  warfare  threat  date,  bases,  and  (3)  standard  training  device  software 
acquisition  manrtgement  procedures.  The  efforts  will  commence  in  FY  84  and  FY  85. 
Future  initiatives  for  JLC  sponsorship  are  solicited,  and  the  method  for  submitting 
them  to  the  JTCG-STD  is  outlined. 


INTRODUCTION 

In  the  mid-1960s  the  military  services 
became  greatly  interested  in  each  others  acqui¬ 
sition,  supply,  and  maintenance  capabilities. 
Out  of  that  interest  and  cooperation  grew  a 
structure  known  today  as  the  Joint  Logistics 
Commanders,  or  JLC.  The  early  efforts 
involved  such  areas  as  munitions  effective¬ 
ness  and  depot  maintenance  interservicing. 

Other  efforts  have  been  added,  and  many  have 
been  completed.  When  the  1973  oil  embargo 
caused  an  upsurge  in  interest  in  training 
simulators,  the  JLC  formed  a  group  to 
coordinate  activities  in  the  realm  of 
training  equipment. 

'The  purpose  of  this  paper  is  to  famil¬ 
iarize  the  training  equipment  community 
with  the  group  that  has  evolved  from  those 
early  ejforts  and  to  solicit  ideas  on  how 
the  services  can  improve  training  effec¬ 
tiveness  through  interservice  cooperation.. 
Effective  interservice  initiatives  can 
benefit  the  operating  commands,  the  "  \ 


acquisition  and  support  organizations,  the 
training  industry,  and  ultimately,  the 
taxpayers. 

THE  JOINT  LOGISTICS  COMMANDERS 

The  Joint  Logistics  Commanders  (JLC)  consist 
of  the  commanders  of  the  US  Army  Materiel  Develop¬ 
ment  and  Readiness  Command,  Air  Force  Systems  Com¬ 
mand  and  Air  Force  Logistics  Command  and  the  Chief 
of  Naval  Material.  USMC  liaison  is  provided  by 
the  Headquarters  USMC  Deputy  Chief  of  Staff  for 
Installations  and  Logistics.  The  current  members 
of  the  JLC  are  pictured  in  Figure  1. 

While  this  joint  arranges  jnt  has  no  legal 
authority,  other  than  that  delegated  to  each 
Commander  to  accomplish  his  assigned  mission, 
the  JLC  management  concept  has  been  endorsed 
by  the  Chiefs  of  the  Military  Departments, 
the  Service  Secretaries  and  the  Secretary  of 
Defense.  The  JLC  meet  formally  at  least 
quarterly  and  periodically  report  to  the 
Deputy  Secretary  of  Defense,  the  Under-Secretary 
of  Defense  for  Research  and  Engineering  (who  is 
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Figure  1.  The.  Joint  Logistics  Commanders  (1  to  r: 
Ccu  Robert  T.  Marsh,  USAF  (AFSC) ;  A dm  J.  C. 

Will.  { ;cis,  Jr.,  USN  (KMC);  Con  Janos  ?.  Mullins, 
USAF  (AI’LC) ;  Con  Donald  R.  Keith,  USA  (DARCOM); 

Lt  Con  H.  A.  Hatch,  USMC) 

Since  the  JLC  are  not  an  official  body, 
there  lit  no  JLC  staff.  Instead,  each  commander 
has  a  member  of  his  special  staff  who  is 
designated  as  Assistant  for  Joint  Service 
Activities.  These  five  are  known  collectively  as 
the  Joint  Secretariat,  and  they  carry  out 
day-to-day  administrative  activities  on  behalf 
of  the  JLC.  The  JLC  themselves  meet  at  least 
quarterly  with  each  commander  taking  his  turn 
to  host  the  others. 

The  efforts  which  the  JLC  direct  are  carried 
out  through  various  groups  and  panels.  Some  of 
these  are  created  for  specific  short  term  tasks 
and  pass  out  of  existence  in  a  matter  of  months. 
Others  have  a  continuing  mission.  Regardless  of 
the  duration  of  the  task,  each  panel  or  group 
consists  of  a  principal  member  from  each  command, 
alternate  members  as  required,  and  an  invited 
USMC  participant  if  the  subject  is  of  interest 
to  the  Marines.  One  principal  member  serves  as 
chairman.  The  groups  and  panels  report  to  the 
JLC  through  the  Joint  Secretariat.  The  group  of 
specific  interest  to  the  training  community  is 
the  Joint  Technical  Coordinating  Croup  on 
Simulators  and  Training  Devices  (JTCG-STD) . 

THE  JOINT  TECHNICAL  COORDINATING  CROUP  ON 
SIMULATORS  AND  TRAINING  DEVICES 

The  JTCG-STD  was  chartered  originally  in 
1976  to  minimize  duplication  in  research  and 
development  efforts.  The  problem  with  that 
charter  was  that  it  did  not  permit  doing  some¬ 
thing  new  which  would  have  multi-service  payoff. 
During  a  f ive-v ear -point  review  of  progress,  all 
parties  agreed  that  a  potential  existed  for  high 
payoffs  from  interservice  efforts  in  the  simulator 
and  training  device  area.  The  question  was  how 
to  achieve  them. 


Several  principles  were  a  :  ..-red  to  in  laying 
out  the  new  approach.  First,  <  forts  would  nave 
to  be  discrete  and  measurable,  n  keeping  with  the 
philosophy  that  the  way  to  eat  elephant  is  one 
bite  at  a  time.  Second,  implementing  efforts 
would  liave  to  be  done  within  t!  "  framework  of 
existing  management  systems  kij.  •.  as  the  Planning, 
Programming,  end  Budgeting  Syst-m  (PPBS).  Third, 
two  or  more  services  would  hav«  lo  share  in  the 
benefits.  Finally,  priority  w.  Md  go  to  those 
efforts  wLth  the  highest  retun  on  investment; 
return  being  defined  as  a  combination  of  increased 
training  effectiveness  with  rekneed  cost. 

Out  of  these  principles  givw  the  concept  of 
"JLC  sponsored"  tasks.  Each  y<  .r  the  JTCC-STD 
will  review  possible  efforts.  hose  with  the 
greatest  payoff  will  be  review.  :  by  the  Joint 
Secretariat  and  presented  to  t:  JLC.  If  the 
JLC  agree  that  the  tasks  should  be  pursued,  they 
will  adopt  them  as  JLC  sponsored.  The  lead 
command,  with  the  cooperation  of  the  others, 
will  then  enter  the  task  into  the  PPBS  as  a  high 
priority  new  start.  Once  the  system  has  been 
operating  for  several  years,  each  year  will  see 
the  completion  of  some  tasks,  continuation  of 
others,  initiation  of  more,  and  approval  of 
additional  ones. 

All  of  these  principles  are  embodied  in  the 
31  March  1982  JTCG-STD  charter  which  is  reproduced 
in  Figure  2.  Perhaps  the  best  way  to  illustrate 
how  the  JTCC-STD  is  operating  is  to  briefly 
examine  three  JLC-spon sored  tasks  initiated  by 
the.  JTCC-STD. 

JLC  SPONSORED  TASKS 

Standard  DMA  Data  Base  Transformation  Program 

Air  Force  Systems  Command  is  the  lead  command 
for  this  effort,  which  will  be  managed  by  the 
Deputy  for  Simulators  at  Aeronautical  Systems 
Division.  It  is  funded  in  FY  84  and  beyond  and 
the  request  for  proposal  will  be  issued  in  FY  83. 
The  objective  is  to  reduce  the  proliferation  of 
transformation  programs  for  Defense  Mapping 
Agency  (DMA)  digital  data.  Changes  in  the  DMA 
specification  would  be  more  easily  accommodated, 
and  data  bases,  whether  radar,  visual,  or  infra¬ 
red,  would  be  transportable  among  training  devices. 
Furthermore,  data  base  enhancements  developed  for 
a  Navy  simulator  could  be  used  directly  for  an 
Army  or  Air  Force  simulator. 

Library  of  Standard  Electronic  Warfare  Threat  Data 
Bases 

Naval  Material  Command  is  the  lead  command 
for  this  effort,  which  will~be  managed  by  the 
Naval  Training  Equipment  Center.  It  will  be  an 
FY  85  start  with  preliminary  activities  in  FY  84. 
The  objective  is  to  be  able  to  produce  or  update 
EW  threat  models  once  and  then  distribute  the 
model  to  all  users.  Whether  this  will  be  done  by 
use  of  a  standard  specification  or  a  model 
librai'y  Is  yet  to  be  determined. 

Standard  Training  Device  Software  Acquisition 
Management  Procedures 

US  Army  Materiel  Development  and  Readiness 
Command  is  the  lead  command  for  this  effort,  which 
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CHARTER 

FOR 

JOINT  DARCOM/NMC/AFLC/AFSC  COMMANDERS 
JOINT  TECHNICAL  COORDINATING  GROUP 
ON 

SIMULATORS  AND  TRAINING  OEVICES 


I.  PURPOSE. 

A  Joint  Technical  Coordinating  Group  on  Simulators  and  Training  Devices 
(JTCG-STD)  is  hereby  established.  Because  the  services  share  the  same 
population  base  as  a  source  of  trainees,  the  same  industrial  base  as  a 
source  of  training  equipment,  and  the  same  potential  adversaries  as  a 
standard  against  which  training  results  must  be  measured,  there  exist 
numerous  opcortunities  to  coordinate  or  consolidate  programs  in  research 
and  development,  acquisition,  and  operation  and  support  of  training 
devices.  The  purpose  of  the  JTCG-STD  is  to  identify  such  opportunities 
and  to  implement  plans  to  reduce  the  cost  and/or  increase  the  effec¬ 
tiveness  of  military  simulators  and  training  devices. 

II  MISSION. 

The  r.;ission  of  the  JTCG-STD  is  to  maintain  oversight  of  all  activities 
within  the  four  logistics  commands  whicl  involve  research  and  development 
acquisition,  or  support  of  simulators  and  training  devices.  Based  on  this 
knowledge  of  planned  and  on-going  activities,  the  JTCG-STD  will  identify 
specific  projects  for  JLC  sponsorship  whicn  offer  high  pay-off  to  two  or 
more  services  from  consolidating  efforts  into  a  single,  JLC-sponsored 
initiative.  In  those  instances  where  JLC  sponsorship  of  a  joint 
initiative  is  not  warranted,  the  JTCG-STD  viill  insure  coordination 
and  exchange  of  information  between  and  among  the  services  to  minimize  or 
eliminate  duplication  of  effort.  In  addition,  the  JTCG-STD  will  facili¬ 
tate  the  exchange  of  information  such  as  technical  reports  and  contractor 
past  performance  between  and  among  agencies  of  the  logistics  commands  to 
improve  the  efficiency  of  operations. 

HI.  GUIDANCE. 

The  JTCG-STD  shall: 

A.  Be  task  oriented.  As  such  it  will  identify  and  sponsor  discrete  tasks 
with  high  pay-off  to  two  or  more  services.  These  tasks,  or  initia¬ 
tives,  will  be  of  finite  duration  with  measurable  goals. 

B.  Communicate  with  appropriate  groups  and  panels  of  the  Iriterservice 
Training  Review  Organization  (ITRO)  to  reduce  or  eliminate  duplication 
of  effort. 

C.  Coordinate  with  the  Directorate  of  Research  and  Technology,  Office  of 
the  Under  Secretary  of  Defense,  Research  and  Engineering  to  eliminate 
duplication  between  the  JTCG  and  the  Simulator  Technical  Advisory 
Group  (SIMTAG). 

0.  Comnunicate  with  other  major  commands  such  as  TRADOC,  ATC,  and 
CNET  which  are  also  involved  in  training  device  acquisition  to 
maximize  the  benefits  of  interservice  cooperation. 


Figure  2.  JTCO-STD  Charter 
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E.  Encourage  the  use  of  the  annual  Interservice/Industry  Training 
Equipment  Conference  as  a  primary  vehicle  for  the  interservice 
exchange  of  information  among  members  of  the  military  training  device 
conmunity. 

F.  Establish  formal  means  of  communication  between  the  JTCG  and  service 
peculiar  simulator  and  training  device  coordinating  organizations  such 
as  the  USAF  Simulator  Advisory  Group  (SAG). 

IV.  REQUIREMENTS. 

A.  Annually,  in  time  to  influence  preparation  of  the  service  Program 
Objective  Memoranda,  the  JTCG-STD  shall  identify  to  the  Joint 
Secretariat  new  joint  service  initiatives  for  JLC  sponsorship. 
Initiatives  shall  be  documented  in  the  format  of  Appendix  II  to  the 
Handbook  for  Joint  Logistics  Commanders  Panels/Groups. 

B.  Annually,  on-going  JTCG-STO  sponsored  initiatives  shall  be  reviewed 
to  determine  whether  continued  JLC  sponsorship  is  warranted.  If  nc 
the  JTCG-STD  shall  recommend  to  the  Joint  Secretariat  that  JLC  spon 
sorship  should  be  withdrawn. 

V.  ADMINISTRATION. 


A.  Periodic  reports  will  be  submitted  in  accordance  with  the  Handhook  for 
Joint  Logistics  Commanders  Panels/Groups. 

B.  Funding  for  a  JTCG-STD  sponsored  initiative  will  be  programmed  and 
budgeted  by  the  lead  logistics  command  for  that  initiative.  Support 
of  the  initiative  by  the  other  logistics  commands  will  be  from 
existing  resources. 

C.  JTCG-STD  initiatives  approved  by  the  JLC  will  be  supported  by  the 
logistics  commands  to  the  extent  necessary  to  assure  that  the  benefits 
of  interservice  cooperation  will  be  realized. 

D.  Lead  command  responsibility  for  JTCG-STD  initiatives  will  be 
equitably  distributed  provided  sufficient  expertise  exists  within  a 
command  to  effectively  lead  the  effort. 

E.  Subgroups  may  be  established  from  time  to  time  with  the  specific 
approval  of  the  OPRs  in  accordance  with  the  Handbook  for  Joint 
Logistics  Conmanders  Panels/Groups. 


F.  In  order  to  identify  areas  for  coordination  or  consolidation  of 
efforts,  the  JTCG-STD  may  review  planned  and  on-going  simulator  and 
training  device  programs  at  meetings  of  the  JTCG. 


G.  Adequate  resources  including  travel  funds  will  be  allocated  to 
JTCG-STD  activities  to  permit  timely  and  effective  operations. 


DONALD  R.  KETTH 
General,  USA 
Conmanding 

U.S.  Army  Materiel  Development 
and  Readiness  Commend 


J.  G.  WILLIAMS,  JR. 
Admira1,  USN 
Chief  of  Naval  Material 
Naval  Material  Command 


JAHES  P.  MULLINS 
eneral ,  USAF 
Conmander 

rAir  Force  Logistics  Command 


ROBERT  T.  MARSH 
General,  USAF 
Commander 

Air  Force  Systems  Conwand 


DATE:  31  March  1982 
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JTCG-STD  INITIATIVE  FOR  A  STANDARD 
DMA  DATA  fcASE  TRANSFORMATION  PROGRAM 


PROBLEM 

A  new,  unique  transformation  program  has  been  procured  for  every  simulator  and 
training  device  whose  radar  or  visual  system  uses  Defense  Mapping  Agency  (DMA) 
data.  Worse,  when  DMA  changes  the  specification  to  which  it  digitizes  world¬ 
wide  data,  each  previously  procured  transformation  program  must  be  revised  to 
accept  DMA  data  which  rreets  the  new  specification. 

BACKGROUND 

The  Defense  Mapping  Agency  integrates  a  variety  of  data  sources  to  produce 
digitized  data  base.  This  data  base  is  used  in  training  devices  to  present  the 
outside-the-cockpit  scene  to  the  aircrew  as  visual,  ground-mapping  radar,  infra 
red,  or  other  sensor  information.  However,  the  ones  and  zeroes  which  compose 
the  digitized  data  base  cannot  be  presented  directly;  they  must  be  manipulated 
to  return  them  to  a  real  world  form.  A  data  base  transformation  program  per¬ 
forms  this  function. 

Ary  data  base  which  DMA  produces  is  formatted  and  detailed  according  to  the  spe 
cification  in  effect  at  the  time  of  production.  Any  change  in  that  specifica¬ 
tion  requires  changes  in  the  transformation  programs  which  will  be  used  to 
process  the  data  base.  The  specification  has  beer,  changed  every  three  or  four 
years  on  the  average. 

DISCUSSION 

There  are  no  theoratical  reasons  why  DOD  could  not  maintain  a  single  data  bast 
transformation  program.  The  government  could  then  provide  the  program  to 
training  device  manufacturers  as  GFE  and  could  revise  it  concurrently  with  data 
base  specification  changes.  Thus,  the  services  would  no  longer  be  paying  for 
multiple  programs  and  multiple  updates.  Furthermore,  some  of  the  products  them 
selves  (for  example,  enhancements  to  the  DMA  data  base)  would  then  be  transpor¬ 
table  between  training  devices. 

In  practice,  the  number  of  data  base  transformation  programs  required  might  be 
two  or  three,  depending  on  application  (radar,  visual,  infra-red)  or  device 
sophistication  (part  task  trainer,  full  mission  trainer).  However,  even  a 
reduction  to  two  or  three  data  base  transformation  programs  would  result  in 
substantial  savings  to  DOD. 

CONCLUSION 

An  effort  to  develop  a  standard  DMA  data  base  transformation  program  is 
required . 

ORIGINATOR 

Name,  complete  mailing  address,  phone  number  (AUTOVCN  for  DCD  personnel). 


Figure  3.  Abstract  format 
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will  be  managed  by  the  Project  Manager  for 
Training  Devices.  It  will  begin  as  soon  as 
planning  can  be  completed  since  it  will  be  done 
primarily  within  the  government.  The  objective  is 
to  reduce  costs  to  us  by  allowing  a  contractor 
to  apply  the  same  software  management  procedures, 
reviews,  and  data  items  to  all  the  Defense 
Department  training  device  contracts  in  his  plant. 
The  team  which  undertakes  this  task  v.'ili  coordi¬ 
nate  its  work  with  t. he  JLC's  .Joint  Policy 
Coord inar.  ing  Croup  on  Computer  Resources  Manage¬ 
ment  ( JPCC-ORM)  .  Training  device  soft  ware  an-1 
computers  are  exempt  from  t ho  requirements  applied 
to  embedded  computer  resources,  but  the  JPCC-CRM 
work  is  a  useful  starting  point  for  training 
device  efforts.  Needless  to  say,  the  JTCC-STD 
will  work  with  industry  on  this  task. 

CHALLENGE  TO  THE  TRAINING  COMMUNITY 

Those  thiec  JLC  sponsored  tasks  illustrate 
the  broad  features  of  the  sort  of  efforts  the 
JTCC-STD  is  chartered  to  undertake.  But  the 
JTCC-STD  has  no  monopoly  on  ideas.  The  JTCC-STD 
needs  ycur  help.  If  you  have  an  idea  to  solve  a 
problem  and  it  meets  the  general  criteria 
outlined  in  this  paper,  the  JTCC-STD  wants  to 
know  about  it. 

The  initial  review  requires  only  a  one  page 
abstract  in  the  format,  of  Figure  3.  Based  on 
the  abstract,  the  JTCG-STL  will  return  it 
because  it  does  not  meet  the  criteria,  hold  it 
for  future  sponsorship,  ask  for  additional 
information,  or  recommend  it  for  Jl.C 
sponsorship. 

Send  your  abstract  to  the  appropriate 
address  listed  below. 

If  you  are  Army: 

US  Army  Project  Manager  for  Training  Devices 
DRCPM-l’NI.-;: 

Attn:  Dr.  R  Hofer 
Orlando,  FI.  32813 

If  von  are  Air  Force: 

Ogden  Air  logistics  Center 
00-A1.0/MMF 
Attn:  Mr.  W.  Haugen 
Hill  AFB ,  FT  SiiOb 

If  \ou  are  Navy,  USMC,  or  Defense  Agency: 

Commander,  Naval  Air  Systems  Command 
NAI R-4 1 31 

Attn:  Mr.  J.  Scarclter 
Washington,  DC  20  360 

If  you  are  outside  the  Department  of  Defense: 

Headquarters  Air  Force  Systems  Command 
HQ  AFSC/SDTa 

Attn:  l.t  Col  C.  Winters 

Andrews  AFB,  DC  20334 

CONCLUSION 

As  the  military  services  view  the  world  of 
training,  many  things  look  the  s.»me--becauso  they 
are  tl.e  same.  We  share  the  same  threats.  We 
share  the  same  pool  of  potential  *  minces.  We 


share  the  same  training  device  industrial  base. 

We  share  the  need  to  provide  effective  training 
at  an  optimum  cost.  As  a  result,  the  teirvices 
could  do  much  together. 

The  .Joint  Logistics  Coiumanders  have  accepted 
the  civil  long#  to  do  more  together  in  the  training 
d?vice  area  and  have  chartered  a  Joint  Technical 
jrdinating  Croup  on  Simulators  and  Training 
Devices  to  make  it  happen.  In  turn,  the  JTCC-STD 
needs  the  help  of  the  training  community. 

Together  we  can  achieve  the  goals  the  Jl.C  have 
established.  The  services,  the  industry,  and 
the  nation  will  be  the  beneficiaries. 
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GETTING  USER  REQUIREMENTS  INTO  THE 
DEVELOPMENT-TO-DEL I  VERY  LOOP 

Frederic  W  Snyder 
Boeing  Military  Airplane  Company 
Wichita.  Kansas 

ABSTRACT 


"iMany  who  coordinate  with  military  users  of  training  systems  are  aware  that  some  trainers  fail  to  satisfy  important  user 
requirement^  Conscientious  peop'e  working  in  the  requirement-to-delivery  Iood  may  never  intentionally  neglect  or  distort 
end-user  n4eds  However,  such  oversight  or  misunderstanding  does  occur  within  the  operations  of  complex  military, 
government,  and  industry  organizations.  The  character  of  formal  and  informal  information  flows  within  and  between  these 
organizations  as  they  relate  to  identification,  establishment  and  communication  of  requirements  hold  the  key  to  improved 
procedures.  What  is  recommended  is  consideration  for  increased  emphasis  on  the  collection  and  use  of  unfiltered  needs 
statements  of  lower  echelon  users  during  the  system  development  to-^elivery  process.  This  envisioned  emphasis  would 
also  include  information  feed-back  loops  to  the  end-user  culminating  in  a  user  orientation  manual  covering  the  intended 
purpose  of  the  trainer,  a  brief  nistory  of  the  system  development,  and  training,  logistics  support,  and  maintenance 
philosophies  . 

/"V 


INTRODUCTION 

;t  is  a  vital  concern  of  the  using  organization  and  the  military  training 
system  contractor  to  know  about  and  understand  the  problems  user 
personnel  are  having  with  a  current  trainer  Unresolved  or  partially 
solved  problems  of  this  sort  lead  to  requirements  for  system  changes 
or.  in  time,  to  replacement  systems  As  an  alternative,  a 
service-oriented  contractor  may  suggest  or  participate  in  a 
reorientation  of  the  approach  to  trainer  use.  maintenance,  and 
logistics  support  that  will  enhance  the  value  or  extend  the  useful  life  of 
the  system  in  close  coordination  with  user  personnel. 

Where  the  system  is  in  current  operation  and  unresolved  problems1, 
lead  to  modified  or  new  systems,  we  can  easily  see  the  validity  for 
strong  inputs  to  requirements  by  lower  echelon  users  However,  when 
a  new  mission  requirement  dictates  a  fairly  radical  change  in  the 
functional  requirement  (e.g  .  advent  of  the  weapon  system  trainer),  or 
when  the  state-of-the-art  enables  the  start  of  a  new  generation  of 
training  systems  (e  g..  the  computer  image  generation  visual  system), 
or  when  cost-ei'ectiveness  of  training  systems  becomes  tne  major 
issue,  then  the  origin  of  the  requirement  comes  from  a  different  pan  of 
the  governmental  or  military  organization  and  it  is  not  so  easy  to 
determine  how  the  ultimate  user  should  be  involved  in  the 
establishment  of  needs,  requirements,  and  specifications  as  well  as  in 
other  phases  of  development  and  evaluations  For  the  potential  user 
and  aH  others  concerned,  there  is  less  information  directly  based  on 
experience  and  more  emphasis  on  trying  to  bridge  between  the  known 
and  the  unknown  This  gap  becomes  wider  the  more  radical  the 
change  from  current  to  future  system  concepts 

The  thesis  of  this  paper  is  that  the  ultimate  user  (instructor  cadre, 
maintenance  personnel,  logistics  personnel)  should  particpate  in 
establishing  and  evaluating  system  needs,  requirements 
specifications,  development,  test,  and  evaluation  for  all  trainers  This 
participation  may  at  times  be  formal  or  informal  It  may  consist  of 
orginatmg  or  reviewing  paper  work,  coordination,  telecons.  gomq  to 
and  participating  m  meetings  and  reviews,  participating  in  tests  and 
evaluations,  receiving  orientation  briefings  tra.nmgs  on  and 
evaluating  new  system  concepts,  being  interviewed  or  answenng 
questionnai.es,  participating  as  a  panei  of  user  experts  to  support 
contractor  design  efforts,  or  participating  as  trainer  ope lators. 
mamtamers  or  logistic  support  persons  in  research  and  development 
tests 

It  is  realized  that  there  are  many  factors  and  constraints  working 
against  the  user  s  systematic  involvement,  one  of  the  most  important 
being  allocating  the  time  away  from  work  and  the  costs  of  travel  and 
subsistence  Other  important  factors  are  the  constraints  imposed  in 
competitive  bidding  and  competitive  fly  offs,  and  strong  attitudes  of 
independence  by  representatives  of  some  companies  and 
government  agencies  If  greater  involvement  and  input  by  the  lower 
echelon  user  in  the  requirement-to-delivery  process  is  viewed  as  a 
prionty  then  these  and  other  constraints  will  be  viewed  as  a  challenge 
and  a  way  often  will  be  found  to  facilitate  such  participation  One  of  the 


most  critical  phases  in  new  system  concept  formation  and 
development  needing  inputs  from  the  user  is  during  the  prepaiction  of 
the  needs  statement  and  in  the  process  of  requirement  defmlion  and 
authentication.  First  we  need  to  define  what  we  mean  by 
requirement  " 

DEFINING  REQUIREMENT 

"What  is  a  requirement"  may  be  said  to  have  a  double  meaning  in 
the  systems  develooment-procurement  cycie  Firs;,  i'i  means  defining 
the  authentic  requirement  which,  in  one  form  or  another,  can  be  used 
to  blaze  the  trail  toward  adequate  fulfillment  of  a  user  s  need  Second 
it  means  that  the  whole  process  starts  with  a  careful  definition  of 
"W!  iAT  "  WHAT"  (i  e  .  what  is  really  required)  is  the  only  question 
that  should  be  allowed  at  this  stage.  Considering  other  questions  too 
soon  only  confuses  the  requirement  issue  For  example,  when  the 
question  "HOW"  is  brought  up  too  soon,  the  requirement 
development  and  communication  process  becomes  confused 
because  everyone  tries  to  solve  a  problem  that  hasn  t  been  properly 
defined.  The  strongest  urge  seems  to  be  to  hurry  through  the  problem 
definition  in  order  to  get  on  with  the  solution,  only  to  find  after  months 
or  years  of  development  and  even  production,  that  the  solution  does 
not  fit  the  real-world  problem  Perhaps  this  is  so  because  many^more 
people  and  organizations  of  people  appear  better  equipped  to  solve 
problems  than  to  ask  questions  and  systematically  and  correctly 
define  problems 

Sometimes  these  two  different  groups  oppose  one  another  when 
they  should  cooperate  to  better  define  and  fulfill  a  requirement 
Typical  arguments  used  on  one  side  are  Stop  wasting  effort  on  an 
ill-defined  problem."  or  on  the  other  We  re  wasting  too  much  effort 
on  understanding  and  defining,  lets  get  on  with  a  solution  Some 
argue  for  a  middle  ground  which  places  both  sides  in  competition  with 
checks  and  balances  but  that  tends  to  stretch  things  out  Soon  the 
schedule  is  tight,  the  budget  is  shert.  and  there  is  no  room  for 
redefinition  of  a  problem,  only  battering  attempts  to  try  to  solve 
whatever  seems  to  be  the  requirement 

What  is  recommended  s  that  the  important  term  requirement  be 
defined  and  communicated  The  requirement  analysis  should  net  be 
contaminated  with  des.gn  solutions  even  if  those  who  are  establishing 
the  HOWs  stadto  work  to  establish  feasible  system  solutions  during 
the  same  time  as  those  who  are  still  tryinq  to  establish  WHATs  (or 
requirement^).  This  is  the  ideal  time  to  utilize  inputs  from  lower 
echeion  users  who  presumably  are  not  design  engineers  or 
requirements  specialists  and  should  be  encouraged  to  think  in  terms 
of  requirements  and  alternative  broad  functions  that  need  to  be 
accomplished  to  fulfill  the  requirements 

THE  AGENCY  REQUIREMENT 

In  order  to  view  the  full  pctenfnl  of  opportunities  to  get  user 
requiremer  :s  into  the  trainer  development-to-delivery  cycle,  we  next 
look  at  the  process  by  which  the  agency  requirement  is  orgmated 
coordinated  and  fulfilled  by  development  of  a  new  system 
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Government  policies,  regulations  and  procedures  tend  to  create  a 
check  and  balance  approach  and  deliberately  involve  all  affected 
parties  to  establish  and  authenticate  adequate  requirements,  which 
when  fulfilled,  will  satisfy  the  assigned  mission(s)  of  the  agency, 
Forms  have  to  be  filled  out.  signed,  and  transmitted  through  the 
proper  channels.  Suggestions  for  improvement  are  studied  which  may 
die  or  be  revised  and  transmitted  to  another  format,  perhaps  as  a 
statement  of  need  Many  are  tabled.  Surviving  requirements  are 
funded  for  further  study,  preliminary  development  and  specification 
wiriting.  If  additional  funding  is  authorized  and  if  industry  is  to 
participate,  a  request  for  proposal  is  prepared. 

Typically,  while  all  of  the  official  actions  described  in  the  policies, 
regulations,  and  procedures  are  taking  place,  many  unofficial  actions 
are  occuring.  The  birth  of  a  firm  requirement  o*ten  takes  a  lot  of  time. 
Anyone  who  has  carried  the  banner  for  a  new  requirement  knows  how 
difficult  it  is  to  find  success  Banner  carriers  are  essential  and  may 
encounter  strong  attitudes  and  feelings,  such  as  the  status  quo  - 
"Don't  rock  the  boat"  and  Let's  make  do  with  what  we  have.  There 
are  memos,  phone  calls,  and  trips  But  establishing  a  firm  requirement 
for  an  important  mission  is  worth  the  effort. 

Not  all  of  the  developmental  effort  is  expended  within  the 
governmental  agencies  If  the  requirement  is  expected  to  result  in  a 
contracted,  funded  development  program,  industry  begins  parallel 
efforts  to  try  to  understand  and  even  participate  in  the  establishment 
oi  the  requirement.  Studies  are  performed  using  research  and 
development  (R&D),  bid  and  proposal  funds,  or  under  contract  to  the 
agency.  Frequent  contacts  are  made  with  user,  requirement  R&D. 
procurement  and  finance  personnel  in  the  agency  or  support 
organization. 

Now  this  sounds  like  a  good  v,  to  establish  useful  requirements 
and  it  works  well  when  there  are  skillful  coordinators  and 
communicators  and  "banner  earners  dedicated  to  understanding, 
defining  and  fulfilling  basic  user  needs.  An  example  of  what  can 
happen  is  that  an  influential  person  or  group  can  favor  a  certain  kind  of 
new  system.  The  system  technology  is  on  the  cutting  edge  of  the 
state-of-the-art.  and  still  under  development  It  does  not  yet  meet  the 
user  required  specification  and  degrades  training  capability. 
Nevertheless  it  is  the  favored  solution  The  users  may  not  have 
sufficient  authority  to  demand  the  system  that  best  fits  their  needs  In 


specific  instances  there  may  be  justifiable  reasons  for  overriding  user 
requirements  just  as  there  may  be  justifiable  reasons  why  some  users 
persist  in  demanding  that  their  training  equipment  requirements  be 
met.  What  is  suggested  here  is  that  there  be  a  fair  and  hopefully 
somewhat  impartial  arbiter  of  honest  differences  The  procurement 
system  already  has  the  checks  and  balances  intended  to  resolve  such 
problems 

COMMUNICATING  AND  THE  INDUSTRY  ROLE 

The  simplified  communication  process  of  a  requirement  as  it  is  done 
within  the  needing  agency  may  look  something  like  what  is  shown  in 
Figure  1 .  The  details  including  iterations  will  vary  depending  on  such 
things  as  procedures,  where  the  need  orginates.  the  priorities,  the 
complexity,  cost  and  length  of  the  program  required  to  fulfill  the  needs, 
and  how  much  opposition  is  experienced 

Industry  representatives  are  sometimes  allowed  to  participate  in 
this  process  They  are  used  as  consultants.  They  do  studies  They 
provide  information  for  agency  presentations  They  help  make 
presentations  to  establish  firm  requirements.  They  travel  and  they 
convey  messages. 

The  way  the  agency  has  of  communicating  within  is  by  internal 
instruments,  such  as  the  Justification  of  Major  System  New  Starts 
(JMSNS)  submitted  with  the  Program  Objectives  Memorandum 
(POM)  A  budget  line  approved  by  Congress  and  a  Program 
Management  Directive  (PMD)  are  confirmation  of  a  firm  requirement 
as  well  as  providing  the  means  of  meeting  the  requirement  Certain 
aspects  of  these  internal  messages  are  made  known  to  industry 
through  briefings  or  other  means.  The  official  message  of  a  beginning 
procurement  process  is  usually  made  in  the  "Commerce  Business 
Daily  "  but  if  that  is  the  first  time  a  company  learns  of  the  requirement, 
it  is  usually  too  late  for  them  to  respond  effectively 

The  notice  in  the  "Commerce  Business  Daily”  is  valuable  and  for 
legal  reasons  essential  in  the  procurement  process  However, 
industry  specialists  who  have  been  working  on  a  pre-proposal  effort 
for  months  have  been  known  to  read  a  notice  so  veiled  that  they  did 
not  recognize  the  item  as  related  to  their  effort  What  is  recommended 
is  a  more  specific  and  revealing  notice  pre-evaluated  for  language 
c'arity  and  simplicity 
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'he  value  of  user  and  procuring  agencies  and  se'ected  industry 
representatives  nteracting  during  requirements  definition  and 
development  are  obvious  The  mGSt  obvious  is  the  expertise  that  can 
be  brought  to  bear  on  development  of  viable  requirement  pros  and 
cons  from  an  independent  source  Although  one  school  of 
requirement  definition  says  requirements  should  be  defined  apart 
from  the  effort  of  obtaining  solutions,  there  is  almost  always  a  vigorous 
interaction  between  requirements  and  the  means  of  meet'ng  those 
requirements  For  example,  among  the  first  thoughts  a  manager  has 
regarding  a  proposed  requirement  is  how  much  will  it  cost9  How  long 
wili  it  take9  This  means  feasible  solutions  as  an  integral  part  of  a 
requirements  definition  and  authentication  are  looked  at  very  early 
Ordinarily,  industry  is  regarded  as  the  prime  inventors  and  doers  of 
the  solutions  intended  to  meet  such  requ,i  aments  Industry  has 
preliminary  design  capabilities,  including  requirement  analysts  who 
are  knowledgeable  and  can  be  tasked  to  work  closely  with  the  agency 
in  defining,  developing  and  selling  a  significant  requirement  But  just 
because  a  liaison  has  been  established  does  net  mean  that  all  is  well 
in  the  communication  process 

Traditionary  the  tendency  is  for  coordination  to  occur  between 
industry  and  agency  perscnnel  at  similar  echelon  levels.  Although 
Figure  2  undoubtedly  oversimplifies  the  fact,  it  makes  a  point  Figuie 
2a  shows  that  in  industry  and  military  agencies,  the  presidents  and 
vice  presidents  tend  to  talk  with  the  generals,  the  chief  engineer  talks 
with  the  colonels,  the  project  engineer  tends  to  talk  with  the  majors 
and  captains  and  the  engineers  talk  with  the  lieutenants  and 
sergeants,  etc  The  industry  marketing  man  or  the  prog  am  manager 
usually  lias  more  flexibility  but  tends  to  talk  with  higher  level 
management  The  specific  levels  involved  are  usually  determined  by 
the  size  and  importance  of  the  proqram  or  potential  program  that 
affects  the  rank  oi  the  management  position  established  to  heat,  un 
the  proje^*  Something  could  be  missed  unless  all  levels  are  at  least 
aware  of  ah  data  in  the  requirements 

Although  perhaps  somewhat  exaggerated  Figure  2b  shows  the 
recommended  approach  to  establish  and  maintain  system 
requirements  and  development  communications  Informal 
communication  wotks  this  way  if  the  freedom  is  allowed  at  least  to 
some  extent  Although  there  are  many  who  would  feel  that  the 
requirement  for  discipline  and  order  in  an  organization  would  be 
compromised  by  such  activity,  thnue  who  have  risked  experimentation 
of  this  method,  even  to  a  limited  extent,  see  merit  in  this  approach 


Whether  official  communication  of  th*s  sort  ts  to  be  allowed  depones 
on  the  people  and  the  operating  procedures  in  the  organization  There 
are  places  in  both  military  and  industrial  organizations  whs-e  it  has 
been  observed  working  well,  at  least  in  an  informal  mode 

CRITIQUING  THE  REQUIREMENT 

An  agency  procedure  for  definng  and  authe.  nesting  firm 
requirements  should,  and  often  does,  take  into  account  that  differing 
and  even  competing  points  of  v.ew  should  be  considered,  studied, 
discussed,  and  hard  decisions  made  Basic  technical  requirements 
are  tempered  by  such  practical  matters  as  priorities,  availability  of 
money,  manpower  and  facilities,  opinions  of  authorities,  ard 
competition  All  requirements  go  through  a  process  of  antique  a^d 
modification  before  finalization 

For  example,  for  many  years  there  has  been  a  technical 
requirement  for  motion  in  flight  simulators  for  certain  mission 
segments  such  as  takeoff  and  landing,  but  only  in  recent  years  has  the 
effectiveness  part  of  the  cost-effectiveness  formula  taken 
precedence  Because  of  the  cost  consciousness  aspect  of 
procurement  in  a  highly  inflationary  period,  even  the  definition  of  the 
cost-effective  procurement"  seems  to  have  changed  The  original 
idea  was  to  achieve  some  kind  of  balance  between  the  cost  of  a 
system  and  the  effectiveness  produced  by  the  system  Some  consider 
the  cost  all  that  really  matters  but  I  don  t  believe  it  In  my  home  we 
have  a  rule  — don't  buy  the  cheapest  product  if  no  one  will  use  it. 
Perhaps  part  of  the  problem  is  that  the  buyer  and  user  are  not  in  the 
same  "household  The  buyer  may  not  know  if  what  he  buys  is 
ineffective  or  even  unusable  and  he  may  never  find  out  because  on 
programs  with  long  developments,  many  times  the  originators  and 
"players'  transfer  to  other  jobs  before  'he  end  item  is  delivered 

Experience  indicates  that  no  statement  of  need,  no  required 
operational  capability,  no  functional  O’  even  detailed  specification  is 
ever  absolutely  complete  This  frailty  in  communication  is  usually  not 
discovered  until  rather  late  in  the  game,  perhaps  during 
hardware-software  integration  or  system  test  and  evaluation,  or 
sometimes  not  until  operational  testing  and  eventual  use  These 
sensitive  to  this  problem  should  allow  for  this  The  recommendation  is 
that  after  doing  all  we  can  to  commumcaie  our  requirements  and 
specifications  adequately,  we  build  in  feedback  systems  to  check  out 
the  responses  of  the  receiver  to  the  senders  message  and  see  if  the 
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Figure  2.  Agency  -  Industry  Communication  Modes  (Conceptual) 


receiver  understands  the  message.  This  involves  far  more  than 
merely  asking  the  receiver  Do  you  understand?"  and  hearing  his 
response.  Yes  ’' 

AVENUES  FOR  IMPROVEMENT 

Let's  next  explore  an  analogy,  a  training  system  development 
approach,  and  topics  on  assuring  trainer  fidelity  and  usability  ihat 
focus  attention  on  the  process  of  defining  and  getting  user 
requirements  into  the  development-to-delivery  loop  It's  not  that  we 
understand  precisely  what  goes  on  and  that  we  can  put  our  finger  on  a 
solution.  We  just  know  that  this  loop'  is  a  very  complex  problem 
formulation  and  communication  process  usually  involving  many 
people  over  a  period  of  time 

1.  The  Open  Shop  Approach 

The  difference  between  the  closed  shop"  and  the  open  shop 
as  it  relates  to  a  customer  getting  what  he  s  ordered  may  help  to 
further  define  the  problem  and  suggest  another  avenue  for 
solution.  In  the  "closed  shop  procedure,  you  write  up  /cur  order 
as  best  you  know  how  and  slip  it  in  the  slot  on  the  shop  door  that 
says  "incoming  orders  Then  you  wait  and  what  vou  thought 
you  ordered  is  delivered  through  a  chute  marked  orders 
delivered  here  " 't  may  or  may  not  be  what  you  expected  but  you 
are  obliged  *o  use  it  the  best  vou  can  Sometime  later  you  are 
allowed  to  make  changes  or  even  to  try  all  over  again 

in  the  "open  shop  you  can  enter  and  talk  with  someone  inside 
until  you  both  are  confident  on  an  agreement  of  requirements 
and  expected  product  capabilities  to  meet  those  requirements 
In  the  open  shop"  you  are  free  to  participate  and  car  even 
influence  the  development  of  the  product  The  product  will  still  be 
the  result  of  compromises,  and  will  not  do  everything  you  would 
like,  but  bv  means  of  personal  representation  your  highest 
priority  requirements  are  usually  met  in  a  cost-effective  and 
timely  wav  and  you  certainly  are  not  surprised  bv  what  you  get 
upon  delivery  The  open  shop"  approach  assures  that  the 
original  messages  of  ultimate  user  needs  are  not  garbled 
twisted  or  lost  If  the  message  does  not  appear  clear,  seems 
exaggerated  or  even  unrealistic,  a  discussion  is  arranged  with 
the  originator  Once  we  have  a  clearly  written  originator  needs 
statement,  it  should  be  earned  along  with  all  endorsements  or 
rebuttal  statements  cf  superiors,  consultants,  engineers 
requirements  people,  procurement  specialists  finance 
specialists  or  anyone  who  would  propose  approval,  disapproval, 
or  changes  to  the  original  statement  All  would  include  reasons 
or  proof  of  their  statements  as  applicable  All  who  see  the 
requirement  in  succession  would  see  the  whole  file  of 
communications  At  imponant  decision  junctures  in 
authenticating  a  requirement  coordination  with  ns  many  of  the 
pai.icipants  in  the  process  as  appropuate  and  piacticable  v'ouid 
be  carried  out.  especially  with  the  originator  This  type  of 
coordination  would  not  preclude  systematic  studies  of  related 
needs  throughout  the  agency  involved  which  might  be  satisfied 
with  a  single  development  and  procurement  program 

2  Adapting  the  iSD  Approach 

The  Instructional  System  Development  (ISD)  process  is  an 
attempt  to  establish  grass  roots  training  requirements  in  a 
systematic  manner,  relying  more  on  a  detailed  analysis  ot  *he 
task  to  be  trained  rather  than  on  the  opinions  of  expert  trainers 
and  their  trainees  Perhaps  not  all  tisks  require  a  detailed 
analysis  it  carefully  planned  *nd  sys!emat*caiiy  obtained  expert 
opinion  can  drlcrm.i  hj  what  the  cnticai  ta>«.s  are  in  training  and 
emphasize  detailed  analysis  only  ot  these  Expert  opinion  on  all 
aspects  of  trzmwg  requirements  should  be  obtained  and 
compared  *  h  znalyt-cally  derived  requ.rements  in  the  ISD 
process  with  .are  taken  not  tc  influence  tfie  one  with  the  other 
loo  ear*y  tn  the  requirement  definition  process 

It  should  be  recognized  that  grass  roots  includes  the  working 
level  as  v.eil  as  lower  echelon  management  In  establishing  flight 


training  requirements,  for  example,  we  need  written  and  oral 
statements  trom  the  flight  line  and  training  simulator  instructors 
and  *neir  trainees.  These  statements  can  often  be  obtained  in 
well  planned  interviews  and  discussions  and  through 
questionnaires  wmch  include  open-ended  questions  Getting 
upward-filtered  grass  roots  opinions  is  an  essential  pari  of  the 
requirement  definition  process  but  should  never  be  permitted  to 
supplant  .nvestigation  at  the  subordinate  levels 

3.  Assuring  Fidelity 

An  observation  made  earlier  in  tms  paper  concerned  the 
difficulty  in  including  in  the  writing  ~,!l  of  the  details  m  the 
requirements  and  specif ictions  necessary  to  assuie  a  usable 
'ramer  An  outstanding  example  is  the  present  inability  to  fully 
pecify  airpi  .ne  handling  qualities  requirements  for  flight 
trainers  The  contractor  understands  this  problem  and  plans  a 
period  for  tuning  «he  system  after  the  system  has  been 
qualified  u^ing  all  availat 'e  specifications  and  aophcabie 
airplane  source  da*a  as  a  guide  In  our  experience,  qualified 
pilots  are  seldom  completely  satisfied  with  the  flight  simulator 
handling  quai'ties  on  the  first  flight  even  though  the  system 
may  have  pas  sett  al1  other  system  specification  tests  And  when 
the  custom**,  test  pilot  flies  »he  flight  simulator  he  sorr  ut'mes 
does  not  like  how  it  handles  At  this  point,  a  period  of  time  will  be 
required  to  fine  tune  the  system  to  a  baseline  acceptable  to  the 
customer  pilot. 

What  should  be  recognized  'S  (U  the  inability  to  fully  specify 
handimg  qualities  is  a  current  state-of-the-art  limitation,  and  (2) 
unti1  overcome,  it  is  recommended  that  this  limitation  be 
recognized  in  the  acceptance  test  procedure,  the  schedule  and 
the  program  budge: 

4.  Assuring  Tratner  Usability  to  Train 

The  present  difficulty  m  fully  specifying  the  use  to  which  a 
system  will  be  put  is  perplexing  and  persists  to  seme  extent  even 
when  ISD  procedures  have  been  in  effect  Given  serious  voids 
the  contractor  may  (1 )  guess.  (2)  ask  a  lot  cf  questions  of  the 
customer  procurement  personnel,  which  he  is  reluctant  to  do  fer 
a  variety  of  reasons  <3j  conduct  company  (usually  abb-eviated  i 
training  requirements  analysis  and  display  the  results  to  the 
customer  for  comment  and  possible  dialogue  on  this  important 
subject.  (4)  visit  and  talk  with  customer  user  personnel  and  ISD 
specialists  about  their  training  requirements  (but  they  may  not 
fully  understand  m  terms  of  a  nev -concept  trainer).  <5i  hire 
highly  qualified  former  customer  use'  personnel  (rf  poss  l  e)  to 
auvise  contractor  management  and  engineers  in  their 
preliminary  design  proposa'  md  full-scale  development  effort.  or 
(6)  use  a  combination  oi  those  and  other  approaches 

Because  training  effect'veness  is  an  elusve  factor,  it  is 
recommended  that  agency  and  industry  work  harder  trying  to 
specify,  understand,  and  comply  wth  the  specifications  Until  the 
sfute-cf-the-ari  allows  for  more  adequate  specifications,  per  naps 
ar.  acceptance  tuning  procedure  agreed  »o  by  both  the 
procuring  agency  and  the  contractor  is  needed  that  can  be  used 
to  establish  more  exactly  what  the  user  needs  m  interaction  with 
the  3ctual  training  device  Obviously  some  flexibility  in  the 
systems,  such  as  in  the  instructional  system,  will  be  required  to 
enable  tuning  to  the  needs  o»  the  instructor  pilot  It  is  'u^her 
recommended  that  consideration  also  be  given  on  se'ectecs 
programs  to  have  a  user  representative  or  panei  on  s.!e  of  the 
contractor  during  both  development  and  test  evaluation 

ALTERNATIVE 

Communicating  is  a  drfficuft  and  can  be  very  frustrating  Care  must 
be  taken  that  frustrations  do  not  take  on  a  persona*'  nature  when  a 
new  k  ok  at  the  whole  process  is  more  in  orde*  T  ake  a  situation  where 
a  customer  knows  m  hrs  own  mind  what  he  wanted  his  new  system  to 
do  and  had  h*gh  expectations  only  to  discover  tar  less  capacity  man 
he  expected  dunng  acceptance  tests  O  'mag*ne  the  frustration  of 
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cons'ienticus  company  people  who  really  wani  ;o  satisfy  a  cummer 
but  cacx)t  seem  to  even  though  the  system  meets  formal 
speatication  requirements  The  usual  trade-ofl  is  eitner  a  dissatisfied 
customer  who  in  time  finds  some  way  to  use  the  new  trainer,  or  a 
series  of  engmeeung  changes  to  improve  the  syrtem  if  the  customer 
can  afford  them 

To  alleviate  the  frustration,  we  need  to  come  to  grips  with  certam 
basic  problems 

1)  Recognize  the  need  for  improve  g  the  requirement 
specification  communication  process 

2}  Define  and  establish  an  approach  to  accomplish  the 
needed  improvements 

3)  Where  necessary,  accept  current  Imitations  in  statements 
of  requirements  and  specifications  and  build  in  feed-back 
loops  to  close  the  communication  information  gap 

4)  Correctly  assess  the  source  of  apparent  human  and 
organization  limits  to  communicate  requirements  and 
specifications  and  assign  alternative  methods  in  both 
plannmq  and  development  to  overcome  communication 
weaknesses 

5)  As  part  oi  the  user  maintenance  manua.s  prepare  a  final 
feedback  on  the  whole  development  process  that 
summarizes  the  contractors  interpretation  of  contract 
specifications,  how  they  were  met.  and  operating  guides 
for  thp  Lse>  to  use  to  attain  the  performance  intended  by 
the  original  requirements  Perh'  ps  tne  manuals  could  even 
precede  delivery  of  the  trainer  equipment  so  surprises 
would  be  minimized  and  full  operational  use  expedited 
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ABSTRACT 

■A 

The  usefulness  or  the  image  produced  bv  a  CIG  system  is  not  well 
cha racter i zed  by  just  the  number  of  edges  or  surfaces  displayed,  but  is.  a 

strong  function  of  the  effectiveness  with  which  the  scene  details  provide 
visual  cues.  This  paper  presents  some  examples  of  what  can  be  achieved 
using  hardware  capabilities  and  modeling  techniques  to  enhance  the  ability 
of  the  CIG  to  present  useful  scene  detail. 

Previous  CIG  systems  changed  scene  details  when  their  image  size  was 
^nall  enough  so  as  not  to  be  distracting  to  the  observer.  Recently 

introduced  system  capabilities  allow  scene  details  to  evolve  in  a  more 
continuous,  smooth,  and  independent  manner.  By  using  these  capabilities, 
details  need  not  be  included  in  the  scene  until  they  are  of  visual 
importance.  A  hierarchical  management  structure  is  utilized  to  provide 
efficient  data  base  cull'.ng  and  level  of  detail  control..  This  enables  data 
bases  with  thousands  of  square  miles,  man;*  levels  of  -dUtaii,  and  thousands 
of  surfaces  per  square  mile  to  be  processed  efficiently  by  the  image 

generator.  Such  data  bases,  rich  in  two-  and  three-dimensional  textural 

features,  can  be  efficiently  produced  by  using  automated  generation 
procedures  that  require  a  minimum  of  modeler  effort. 


INTRODUCTION 

The  imagery  produced  by  a  real-time 
computer  image  generator  (CIG)  is,  in 
truth,  a  very  sophisticated  illusion. 
The  participant  in  a  simulation  exercise 
should  feel,  think,  and  react  as  if  what 
he  is  seeing  is  real,  when  in  fact  it  is 
not.  As  the  range  of  training, 
engineering,  and  test  environments 
broadens  and  the  demand  on  training 
effectiveness  increases,  so  does  the 
demand  on  the  CIG. 

The  problem  for  CIG  systems  has 
^always  been  to  produce  the  "most 
effective  image"  possible  within  a  set 
of  constraints.  The  typical  measure 
used  to  define  the  "most  effective 
image"  has  been  the  number  of  edges  or 
surfaces  that  can  be  displayed.  This 
has  proven  to  be  an  inadequate  metric  in 
specifying  the  capability  jf  a  CIG 
system.  The  effectiveness  of  the 
illusion  is  more  dependent  on  the 
system's  ability  to  provide  sufficient 
cues  for  a  particular  task,  whether  it 
be  high  altitude  navigation  or  low 
altitude  weapons  delivery.  This  ability 
can  be  reasured  in  terms  of  the  number 
of  displayed  edges  or  surfaces,  the 
number  and  density  of  cues,  the 
concentration  of  cues  where  they  are 
most  important,  and  the  system's  ability 
to  manage  the  data  base  in  a  manner  that 
is  both  non-di stracting  to  the  observer 
and  efficient  in  presenting  scene  detail. 

To  enhance  the  „■  ossputer-generated 
illusion,  an  effective  combination  of 
modeling  strategies  and  image  generator 
capabi  1  i  t.  i  es  is  required.  The 


development  of  the  techniques  presented 
in  this  oaper  was  done  on  an  Evans  & 
Sutherland  CT."*A  image  generator.  This 
paper  will  introduce  some  of  the  .system 
capabilities,  and  the  modeling 
strategies  which  have  been  developed  to 
more  fully  utilize  these  capabilities. 


HARDWARE  FEATURES 

This  section  will  outline  the 
structure  of  a  CT5A  data  base  and  then 
discuss  some  of  the  features  of  the 
CIG's  cell  processor,  which  does  the 
initial  processing  of  the  data  base. 

Data  Base  Structure 

A  CT5A  data  base  is  a  collection  of 
scene  elements  that  has  been  organized 
into  a  hierarchical  structure  to  allow 
efficient  processing  by  the  image 
generator.  Scene  elements  consist  of 
planar  surfaces,  called  polygons,  and 
lights.  Objects  in  a  data  base  are  made 
up  of  a  collection  cf  these  surfaces  and 
lights.  Usually  an  object  is  thought  of 
as  a  single  entity  such  c.s  a  tree,  bush, 
or  rock.  5y  organizing  surfaces  of 
various  sizes  and  shapes  the  modeler  can 
create  complex  items,  soch  as  aircraft 
ard  shifts.  The  modeler  can  determine 
the  visual  attributes  of  an  object  by 
specifying  the  color  and  shading 
characteristics  of  the  surfaces  that 
comprise  the  object.  CT5A  also  allows 
objects  to  be  made  transparent,  and 
several  levels  of  transparency  are 
provided. 

The  hierarchical  structure  of  the 
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data  base  can  be  likened  to  a  tree  with 
main  branches  that  diverge  into  smaller 
branches  and  eventually  down  to  the 
individual  leaves.  The  CT5A  image 
generator  traces  through  the  d*ta  ba_se 
tree  and  applies  a  series  of  cull 1 ng 
tests  to  determine  which  portions  of  the 
entire  data  base  should  be  considered 
for  further  processing  and  possible 
display.  These  culling  tests  are 
performed  to  eliminate,  as  early  as 
possible,  and  in  pieces  as  large  as 
possible,  those  portions  of  the  data 
base  that  will  not  appear  in  any  display 
channel.  These  culling  tests  include 
level  of  detail  (only  the  simplest 
representation  of  an  object  necessary 
for  a  given  viewing  distance  is  used) 
and  field  of  view  testing  (does  the  item 
impinge  on  any  display  channel).  An 
example  of  this  data  base  tree  tracing 
will  illustrate  this  concept. 

Imagine  a  data  base  which  consists 
of  the  entire  continental  United 
States.  If  the  simulated  e/epoint  is 
positioned  so  as  to  be  looking  directly 
at  the  Evans  fc  Sutherland  building  in 
Salt  bake  City,  Utah,  then  the  image 
generate’  must  trace  through  the  data 
base  tree  for  the  whole  United  States 
and  decide  which  surfaces,  out  of  the 
billions  possible,  should  be  displayed 
to  represent  this  scene.  At  the 
beginning  of  the  tree  processing  the 
entire  continental  United  States  is 
represented  by  what  is  known  as  a  cell. 
A  cell  is  a  volume  in  3  space  which  has 
a  high  and  a  low  level  of  detail  (LCD), 
i.e  ,  *  c  ’'plex  and  a  simple 
representation.  Each  representation  nay 
either  be  an  object,  which  is  output 
directly  to  the  geometric  processing 
function  for  eventual  display,  or  a 
collection  of  cells.  This  collection  of 
cells,  called  a  mesh,  forms  another 
branch  of  the  data  base  tree  which  is 
further  processed  by  the  image 
generator.  Each  cell  also  has  an 
associated  transi tion  range ,  which  tells 
the  image  generator  whicn  LOD  to 
process,  depending  on  the  eyepoint's 
distance  from  that  cell. 

Since  the  simulated  eyepemt  is 
within  the  cell  for  the  United  States, 
the  high  LOD  option  is  pursued  and  all 
processing  on  the  low  LOD  stops.  The 
high  LOD  for  this  United  States'  cell 
may  consist  of  a  mesh  of  48  cells,  where 
each  cell  represents  a  state.  The 
culling  tests  are  applied  to  these  48 
cells  to  determine  that  only  the  cell 
for  the  state  of  Utah  is  within  the 
specified  transition  range  and  needs 
further  processing,  while  the  other  4? 
states  need  no  longer  be  considered. 
The  cell  for  Utah  also  has  a  low  and  a 
high  level  of  detail  with  an  associated 
transition  range.  The  high  LOD  is 
processed  further  by  the  image  generator 
since  the  eyepoir . .  ts  within  the 
specified  transition  ra  ,ge.  This  high 
LOO  is  made  up  of  several  counties  in 


the  state,  with  each  county  being  a 
cells  Again  the  culling  tests  on  these 

county  cells  are  applied  to  find  that 
only  Salt  Lake  County  needs  further 

processing . 

Each  of  the  cities  in  Salt  Lake 
County  .nay  be  considered  a  cell  with  its 
own  high  and  low  level  of  detail.  Since 
the  eyepoint  is  in  Salt  Lake  City,  the 

high  level  of  detail  for  the  city  is 

processed  further  while  the  lower  levels 
of  detail  for  some  of  the  nearby  cities 
may  also  be  processed.  The  Sal..  Lake 

City  cell  may  have  several  subcells  for 
sections  of  the  city,  and  aoain  the 

culling  tests  would  choose  the  East 
Bench  area  for  processing  at  its  high 

LOD,  while  other  portions  of  the  city 

might  have  their  lower  levels  of  detail 
represented  depending  on  their  distance 
from  the  eyepoint.  Tracing  through  the 
data  uase  tree  in  this  marner  continues 
until  the  particular  scene  that  is  being 
observed  is  culled  out  of  the  rest  of 
the  features  in  the  Sast  Bench  portion 
of  the  city. 
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In  this  manner  the  image  generator 
has ,  with  very  few  decisions,  culled 
through  the  entire  United  States  data 
base  to  determine  exactly  which  £ urfaces 
need  to  be  processed  to  display  the 
desired  buildiu>~  2nd  nearby  details. 
This  type  of  hierarchical  structure 
allows  the  CT5A  hardware  to  trace 
through  an  extremely  complex  data  base 
tree  very  rapidly  and  efficiently. 

The  Cell  Pr  ocessor 

The  cell  processor  is  that  portion 
of  the  image  generator  which  traces 
through  the  data  base  tree,  once  every 
field,  to  determine  which  set  of  objects 
should  be  considered  for  further 
processing.  Several  features  in  the 
cell  processor  enhance  its  ability  to 
perform  this  function. 

Perspective  Foreshortening 

Correction.  When  a  primarily  flat  scene 
detail  is  viewed  at  a  very  shallow 
glancing  angle,  its  effective  screen 
subtense  may  be  much  less  than  its 
nominal  perspective  size  based  on  range 
alone.  This  perspective  foreshortening 
means  that  the  displayed  visual 
significance  of  the  item  will  be 
generally  less,  sometimes  much  less, 
than  if  it  were  always  viewed 
straight-on.  CT5A  provides  a  method  of 
accounting  for  the  perspective 

foreshortening  of  items  by  correcting 
the  computed  transition  range  to  delay 
introduction  of  an  object  in>.o  the 
active  scene  to  provide  for  equivalent 
visual  impact  at  the  switch.  This 
process  helps  avoid  tho  compression  of 
details  at  the  horizon,  and  their 
attendant  buildup,  and  further  frees  1G 
resources  for  use  on  more  visually 
significant  scene  details. 

Fade  Level  Of  Detail.  Fade  level 
of  detail  makes  the  transition  between 
levels  of  detail  smoother  and  less 
distracting  to  the  observer.  When  this 
is  used,  the  transition  is  done 
gradually.  Using  the  transparency 
feature,  the  incoming  object  starts  out 
transparent  ana  becomes  more  opaque, 
while  the  outgoing  object  starts  out 
opaque  and  becomes  more  transparent. 
This  gradual  switching  between  levels  of 
detail  allows  the  transition  to  take 
place  much  closer  to  *  he  observer 
without  being  distracting. 

Load  Management .  The  CT5A  employs 
a  harlw-.*?  e-based  load  management  scheme 
which  t-s*es  advantage  of  the  fact  that 
small  modi r icat ions  in  LOD  transition 
ranges  throughout  a  data  base  can  have  a 
Considerable  effect  in  altering  the 
image  generator's  system  load,  without 
having  a  significant  ci  feet  on  the 
visual  scene.  The  transition  ranges  on 
each  cell  in  the  data  base  are  modified 
tc  keep  the  image  generator  running  at 
or  below  the*  nominal  time.  Modifications 
to  th<  transition  ranges  are  done  in 


real  time  and  are  small  enough  to  reduce 
the  system  load  when  necessary  without 
becoming  visually  apparent.  Since  the 
load  management  is  an  automatic  process, 
the  modeler  does  not  need  to  be 
concerned  about  creating  a  data  base 
that  will  never  overload  the  system,  he 
only  needs  to  have  a  data  base  whose 
average  complexity  is  within  the 
system's  limits. 


MODELING  TECHNIQUES 

The  character i zat i on  of  what 
constitutes  1  useful  image  has  always 
been  a  very  difficult  task.  Many 
considerations  must  be  taken  into 
account  which  are  highly  dependent  on 
the  training  task,  the  image  generator's 
capabi 1 i 1 1 es  ,  and  the  subjective  bias  s 
of  those  using  the  system.  There  are, 
however,  several  objective  measures  that 
can  aid  in  determining  the  usefulness  of 
a  data  base. 

1.  How  many  scene  elements  can  be 
displayed  in  each  channel,  and  in  the 
aggregate  channels?  This  number,  which 
is  hardwa re-depenaent,  has  typically 
been  the  measure  of  a  CIG's  capability, 
yet  important  issues  arise  about  how 
those  scene  elements  are  used  to  provide 
cues . 

2.  What  is  the  density  of  scene 
elements  or  objects  in  the  modeled 
environment?  The  density  cf  objects  in 
the  scene  indicates  the  frequency  of 
cues  and  determines  whether  sufficient 
information  can  be  presented  to  perform 
a  particular  task.  Training  tasks,  such 
as  nap  of  the  earth  helicopter  flying, 
may  require  some  type  of  ground  object 
every  25  feet,  while  medium  altitude 
flying  might  require  one  every  200H 
feet.  Some  tactical  missions  require 
appropriate  cues  throughout  a  large 
altitude  range,  implying  a  capability  to 
make  a  graceful  transition  to  addit.onal 
detail  at  lower  altitudes  without 
exceeding  the  system's  capacity  at  the 
higher  altitudes.  The  user  of  a 
simulator  mu  ;t  determine  the  density  of 
cues  needed  fo..  the  task  and  then 
specify  an  image  generator  and  modeled 
data  base  which  can  suppor*  that  density. 

3.  What  is  the  average  density  of 
scene  elements  or  objects  in  the  image 
plane?  Because  of  scene  compression 
near  ^he  horizon,  a  data  base  that  has  a 
unitoim  distribution  of  objects  in  model 
space  will  not  be  uniformly  distributed 
in  the  image  plane,  but  will  tend  to 
have  increased  density  with  distance 
from  the  eye.  This  tends  to  •crowd"  the 
objects  in  a  very  narrow  band  near  the 
horizon,  at  the  expense  of  sparser 
detail  closer  to  the  observer.  Methods 
that  can  create  a  nearly  constant 
density  of  objects  in  the  image  plane 
will  allow  much  more  visually  effective 
use  of  scene  detail. 
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4.  Are  objects  provided  at  various 
oizes  so  that  cues  are  available  at 
several  ranges  from  the  eye?  The  eye 
has  a  tendency  to  ignore  visual  cues 
from  objects  which  are  either  too  large 
or  too  small.  Whatever  the  actual  size 
of  an  object,  there  is  a  range  of 
distances  in  which  the  object  will  have 
the  proper  perspective  size  to  be  of 
visual  significance,  and  outside  of 
which  the  object  will  be  increasingly 
ignored.  This  implies  that  the  data 
base  must  provide  cues  which  are  of 
various  sizes  so  that  those  objects, 
which  are  larger  and  spaced  less 
frequently,  are  mixed  in  with  small 
items  which  are  spaced  closer  together. 
This  provides  a  fairly  constant  spatial 
frequency  of  cues  for  the  participant  in 
a  visual  simulation. 

Several  modeling  techniques  have 
been  developed  to  increase  the  effective 
scene  content  of  a  data  base  based  on 
the  criteria  set  forth  above.  These 
methods  can  provide  ways  of  analytically 
designing  and  then  producing  a  data  base 
which  will  satisfy  a  broad  range  of 
training  tasks. 

Level  Of  Deta i 1 

The  modeler  uses  a  level  of  detail 
strategy  to  create  several 

representations  of  an  object,  eacn  with 
a  different  number  of  scene  elements. 
For  example,  an  airplane  might  have 
three  levels  of  detail,  i.e.,  three 
representations,  one  with  440  surfaces 
for  the  highest  detail,  another  with  140 
surfaces,  and  the  lowest  detail  ntay  only 
be  32  surfaces.  The  image  generator 
chooses  which  representation  to  display 
based  on  the  distance  from  the  eyepoint 
to  the  airplane.  In  creating  the  data 
base,  the  modeler  specifies  the.t  the 
airplane  will  first  appear  as  the  lowest 
level  of  detail  at  3.25  nm.,  then  be 
replaced  by  the  middle  level  of  detail 
at  0.5  nm.,  and  the  highest  detail 
representation  will  be  shown  at  0.1  nm. 

The  modeler  tries  to  specify 
transitions  to  take  place  as  close  as 
possible  without  making  the  replacement 
distracting,  ana  thus  saves  the  image 
generator  as  much  pr  essing  as 
possiDle.  The  distance  at  which  the 
transition  takes  place  is  based  on  suers 
criteria  as:  1;  the  perspective  size  of 
the  object,  2)  the  relative  contrast 
between  the  object  and  its  background, 
3)  the  shape  of  the  object's  silhouette, 
and  4)  the  .elative  geometric 
differences  between  the  levels  of  detail 
which  are  switching.  by  using  simpler 
representations  for  objects  at  greater 
distances,  the  image  generator  has 
greater  capacity  to  display  more 
objects,  thus  increasing  the  number  of 
visual  cues  that  can  be  displayed. 

The  level  of  detail  strategy 
applies  not  only  to  individual  objects 


but  also  to  every  cell  in  the  data 
base.  Each  cell  in  the  data  base  has  a 
low  and  a  high  option  pointer  which  can 
be  considered  the  low  end  ninh  levels  of 
detail  for  that  cell.  This  dlows  an 
entire  region  to  hav*>  a  low  and  a 
detail  representation,  as  well  as  each 
portion  of  the  region.  The  real  Dower  of 
the  strategy  comes  from  the  hierarchical 
nature  of  the  cell.  By  creating  a 
hierarchy  of  cells,  the  modeler  can 
create  a  tree  with  a  high  and  low  level 
of  detail,  then  a  group  of  trees  with  a 
high  and  low  level,  then  a  forest  with  a 
high  and  low  option,  and  even  an  entire 
large  section  of  forested  terrain  with 
noth  options. 

An  example  of  this  can  be  seen  in 
Figure  6.  Represented  in  the  desert 
scene  are  six  levels  of  detail.  The 
first  level  of  detail  consists  of  the 
basic  desert  floor  with  a  few  mesas. 
Nearer  to  the  eye,  buttes  are  added  to 

form  the  second  level  of  detail  and 
complete  the  large  objects.  The  third 
level  of  detail  is  formed  by  adding  sand 
dunes  around  the  mesas  and  buttes. 

These  dunes  are  added  to  the  scene  :t  a 
distance  which  makes  their  silhouette 
visually  significant.  The  fourth  level 
of  detail  is  to  add  the  cacti  on  and 
around  the  dunes  and  buttes.  The  fifth 
level  of  detail  adds  tne  bushes  to  the 
scene.  Rocks  and  pebbles  make  up  the 

sixth  level.  Tf. '  ~  six  level-cf -detai  1 
desig-  of  the  desert  applies  the 
level-of-detai 1  ideas  to  the  desert  as  a 
whole,  as  well  as  to  each  palette  item 

in  the  desert,  and  makes  the  scene  very 
detailed  near  the  eye  bu>_  fairly  simple 
far  away  where  the  small  items  are  not 
important . 

1 ade  Leve 1  Of  Deta i 1 

With  the  hardware  fade 

level-of-detai 1  capability,  the  modeler 
has  greater  leverage  in  specifying  the 
transitions  between  representations  of 
an  object.  Previous  strategies  for 
level  of  detail  switching  required  a 
sudden  replacement  of  one  representation 
for  another.  This  required  the  switch  to 
tc’.ke  place  when  the  perspective  sizes  of 
the  objects,  or  their  relative 
differences,  were  sufficiently  small  so 
as  not  to  be  distracting  tc  the 
observer.  This  was  typically  at  a  much 
greater  distance  than  that  at  which  the 
resulting  details  were  useful.  Thus  a 
significant  amount  of  detail  had  to  be 
processed,  not  because  it  was  useful  but 
solely  to  help  keep  its  introduction 
from  becoming  objectionable. 

It  fade  level  of  detail  has  been 
selected  by  the  modeler  .or  the 
transition  between  an  object's  two 
levels  of  detail  (LOD),  then  at  the 
specified  transition  range  the 
replacement  LOD  is  brought  into  the 
active  scene  as  a  transparent  object. 
As  the  distance  from  the  eyepoint  to  the 
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object  changes  the  replacement  i.OD 
gradually  becomes  more  opaque  while  the 
replaced  LOD  becomes  more  t ra nspa rent . 
When  the  replacement  LOD  is  fully  opaque 
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transitions  to  take  place  closer  to  the 
observer.  The  subtlety  also  enables  the 
use  of  fewer  levels  of  detail,  with 
greater  geometric  differences  between 
levels,  yet  still  without  distraction  at 
transition.  Fifty  percent  reductions  in 
the  transition  range  are  not  unusual. 
This  means  that  simpler  representations 
art  more  often  displayed,  which 
conserves  system  resources  and  allows 
more  objects  to  be  displayed. 

Level  Of  Scale 

A  visually  effective  data  base  must 
provide  j  selection  of  visual  cues  of 
differing  sizes  and  spacing.  Each 
particular  visual  cue  has  a  range  over 
which  it  contributes  significantly  to 
the  image;  beyond  this  range  its 
decreasing  size  makes  it  less  and  less 
valuable,  and  the  modeling  strategy 
should  remove  it  from  the  scene  when  its 
value  drops  below  some  threshold.  To 
extend  the  scene  richness  beyond  this 
range,  other  visual  cues  are  needed 
which  are  substantially  larger  (hence 
continue  to  have  visual  significance  at 
the  further  tanges),  and  are  spaced 
farther  apart.  Several  additional  leves 
of  scale  may  oe  provided,  each  with  a 
particular  range  over  which  it 
contributes  significance,  until  a 
continuity  of  detail  to  the  maximum 
required  range  is  achieved.  The  largest 
levels  of  scale,  which  are  introduced 
into  the  scene  at  the  longest  ranges, 
are  not  removed  from  the  scene  as  the 
ooserver  gets  close  to  them,  but  their 
greater  spacing  keeps  their 

contributions  from  overloading  the  image 
generator.  Sjch  level  of  scale  can  be 
designed  to  requii-  a  percentage  of  the 
overall  image  generator  resource,  and 
then  several  levels  added  together  to 
achieve  the  required  depth  of  scene 
without  image  generator  overolad.  The 
desert  shown  in  Figure  6  uses  six  such 
levels  of  scale,  and  achieves  scene 
richness  from  very  near  the  observer  out 
to  ten  miles.  The  apparent  scene 
density  remains  high  ovei  a  wide  range 
of  flight  altitudes,  and  the  emergence 
of  the  smaller  levels  of  scale  provides 
valuable  velocity,  altitude  and 
time-to-impact  cues. 

Computational  Methods 

{Manning  and  executing  the  design 
of  a  data  base  to  accomplish  specific 
objectives  within  given  image  generator 


constraints  has  tradi tionall y  been  a 
difficult  and  iterative  task.  The  next 
few  sections  of  this  paper  indicate  how 
such  planning  and  estimating  can  be 
successfully  accomplished  during  the 
design  phase,  .and  bow  the  data  base 
structure  makes  it  feasible  to 
modularize  the  modeling  process.  In 
performing  these  computations,  severa1 
approximations  are  made  concerning  the 
data  base  and  rhe  methods  used  in 
deriving  the  results.  The  data  bases 
used  in  these  examples  are  assumed  to  be 
sufficiently  flat  that  the  surface  area 
of  the  terrain  can  be  approximated  by  a 
plane  and  planar  area  formulas  can  be 
used.  When  the  area  under  consideration 
e>Lends  from  the  eyepo:  nt  out  to  a 
particular  distance,  the  area  will  be 
computed  as  a  sector  of  a  circle.  If 
the  area  under  consi dera tion  is  bounded 
by  two  nonzero  distances  from  the 
eyepoint,  then  the  area  will  be  computed 
as  a  sector  of  an  annulus.  The  average 
ground  area  per  object  is  computed  as 
the  total  area  covered  by  the  objects 
divided  by  the  number  of  objects  in  that 
area.  The  average  spacing  between 
objects  is  simply  the  square  root  of  the 
average  ground  area  per  object. 

The  hierarchical  nature  of  the  data 
base  tree  allows  the  modeler  to 
construct  the  data  base  in  modules  which 
can  then  be  concatenated  to  form  the 
whole  data  base.  The  impact  of  e3ch 
module  on  the  image  generator's  capacity 
can  be  computed  independently  from  the 
other  modules  in  the  data  base.  As  the 
modules  ure  joined  to  form  larger 
nodules,  the  result  on  system  load  is 
>dditive.  This  allows  the  modeler  to 
start  with  the  specifications  for  a  data 
base  and  systematically  oudnet  the  image 
generator's  resources  among  various 
yoos  of  detail  to  produce  the  des  i  red 
data  base. 

The  arithmetic  used  in  deriving  the 
results  of  the  next  few  sections  is 
simple  and  str a ight f orwa rd .  Sufficient 
information  is  provided  so  the  reader 
can  verify  the  numbers  if  desired. 

Scene  Element:  Densities  In  Model  Space 

Computing  scene  element  densities 
gives  an  indication  of  the  number  of 
visual  cues  presented  in  a  scene  as  well 
as  their  average  spacing.  If  the  data 
base  is  assumed  to  lie  essentially  in 
the  ground  plane,  then  scene  element 
density  means  the  number  of  scene 
elements  per  square  unit  when  the 
highest  level  of  detail  for  objects  in 
the  area  is  presented,  e.g.,  surfaces 
per  square  nautical  mile  (sqnm).  This 
is  usually  computed  as: 

No.  of  objects  No.  of  surfaces 

Dens i  ty=  - -  *  - - 

sqnm  object 


Because  of  che  limited  capability 
of  an  image  generate!  to  display  scene 
elements,  the  allowable  density  of 
objects  in  a  data  base  is  constrained  by 

the  tnta]  *w’mbe:  "f  ~r,f'r  -1 - . - 

can  be  processed  and  the  simulator's 
field  of  view.  In  the  following 
examples  the  visual  system  is  assumed  to 
have  a  120-degree  horizontal  field  of 
view. 

The  above  relationship  gives  the 
modeler  four  parameters  which  can  be 
adjusted  to  meet  the  needs  of  the 
simulation  problem:  number  of  objects, 
the  area  which  is  covered  by  the  highest 
leve  1  of  detail  of  these  objects,  the 
number  of  surfaces  per  object,  and  the 
density.  Any  three  of  them  necessarily 
define  the  fourth. 

Typically  a  modeler  will  be  faced 
with  a  problem  where  one  or  more  of  the 
parameters  will  be  specified  for  him  and 
he  must  create  the  data  base  to  fulfill 
those  specifications.  A  problem  a 
modeler  might  encounter  is  to  create  a 
forest  which  uses  an  average  of  1500 
surfaces  for  the  trees  within  the  field 
of  view.  The  next  task  the  modeler 
would  do  is  to  design  a  tree  or  several 
types  of  trees  which  he  would  like  to 
put  in  the  forest.  The  modeler  can  make 
a  tree  which  uses  anywhere  from  several 
hundred  surfaces  to  a  simple  three-sided 
pyramid  in  its  highest  LCD;  the  choice 
is  based  on  the  visual  requirements  foi 
the  simulation  exercise.  Some 

situations,  such  as  nap  of  the  earth 
helicopter  flying,  may  require  very 
detailed  trees,  while  other  applications 
for  higher  altitudes  may  be  satisfied  by 
simpler  representations.  The 

specifications  for  the  data  base  should 
indicate  the  types  of  visual  cues 
required.  Suppose  that  in  this  example 

the  forest  will  be  flown  over  at  a  high 
velocity  and  only  fairly  simple 
representations  for  the  trees  are  needed 
to  give  height  and  velocity  cues.  Based 
on  this  criteria  the  modeler  creates  a 
tree  which  has  10  surfaces.  After  some 
experimentation  the  modeler  finds  that 
if  the  tree  is  added  to  the  active  scene 
at  a  range  of  5  nm.  ,  then  the  audition 
of  the  tree  is  not  distracting  to  the 
observer. 

The  modeler  has  now  specified  all 
of  the  parameters  in  the  density 
relationship.  The  area  in  a  120-degree 

field  of  view  sector  of  a  5  nm  radius 
circle  is  26.2  sqnm  with  1500  surfaces 
desired.  This  implies  a  density  of  57 
surfaces  per  sqnm  or  6  trees  per  sqnm. 
This  density  means  that  the  average 
ground  area  per  tree  is  6.44  million 

sqft.  or  an  average  spacing  of  2,500 
feet  between  trees.  The  modeler  can 
then  cieate  his  data  base  so  that  the 
placement  of  trees  in  tht  forest  is  an 

average  of  every  2,500  ft. 

By  using  fade  level  of  detail,  the 


transition  range  at  which  the  tree  is 
added  to  the  active  scene  can  be 

decreased  considerably.  If  in  the  above 
example  the  transition  range  were  to  be 
t  r  ro  1.5  nm. ,  then  the  resultant 

toresc  would  have  an  average  surface 
density  of  637  surfaces  per  sqnm.  or  64 
trees  per  sqnm.  with  the  average  spacing 
between  trees  being  762  ft.  The 
tremendous  leverage  that  is  gained  by 
decreasing  the  transition  range  is  due 
to  the  fact  that  the  total  area 

displayed  at  a  given  LOD  decreases 
proportionately  to  the  square  of  tne 
transition  range. 

Using  multiple  levels  of  detail 
also  gives  the  modeler  tremend^js 
leverage,  since  representations 

requiring  fewer  surfaces  car  be  used  for 
the  sections  of  the  field  of  view  which 
may  cover  large  areas.  Suppose  the 
modeler  finds  that  he  can  create  a  lower 
level  of  detail  for  the  tree  which  uses 
only  3  surfaces.  The  modeler  also 
changes  the  transition  ranges  so  the  low 
level  of  detail  comes  in  at  1.5  nm.  and 
the  high  level  of  detail  comes  in  at 
0.25  nm.  Using  these  parameters,  the 
modeler  computes  that  his  forest  can  now 
have  a  surface  density  of  1,992  surfaces 
per  sqnm.  or  199  trees  per  sqnm.  with  an 
average  spacing  between  trees  of  430  ft. 

Table  1  summarizes  the  results  of 
these  three  examples  and  shows  the 
tremendous  increases  in  surface 
densities  realized  when  using  multiple 
levels  of  detail  and  the  fade  level  of 
Jetail  option. 


Tree 

Conf igurat ion 

Average 
Spacing 
between 
trees  (ft) 

Polygon 

Density 

(surfaces 

/sqnm) 

Tree 

Density 

(trees 

/sqnm) 

1  level  cf 
detail,  5  nm. 
transition 

2,500 

57 

6 

1  level  of 
detail  1.5  nm 
transition 

762 

637 

64 

2  levels  of 
detail,  1.5 
and  0.25  nm 
transi tions 

430 

1,992 

199 

Table  1 

Forest  Tree  Surface  Densities 


Image  Plane  Scene  Element  Derg i ties 

If  the  density  of  objects  in  model 
space  is  constant,  as  in  the  examples 
above,  the  density  of  objects  in  the 
image  plane  will  increase  with  the 
distance  from  the  eye.  This  is  simply 
because  the  area  in  a  given  section  of 
the  image  plane  increases  with  that 
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section’s  distance  from  the  observer. 
This  tends  to  concentrate  the  objects  in 
a  scene  close  to  the  horizon  and  leave 
fewer  objects  close  to  the  eye.  To 
efficiently  use  the  displayed  objects 
for  visual  cues,  the  observer  would  like 
to  have  the  density  of  objects  remain 
constant  in  the  image  plane  so  that  the 
number  of  objects  close  to  the  horizon 
is  nearly  the  same  as  it  is  close  to  the 
eye.  This  implies  that  the  density  cf 
objects  in  model  space  must  decrease 
rapidly  with  distance  from  the  eye. 

When  computing  the  image  plane 
density  the  modeler  needs  to  know  the 
vertical  field  of  view  configuration  for 
a  simulator.  Figure  2  shows  a  possible 
configuration. 


IMA. 3  HAM 


Figure  2 

Vertical  FieU  Of  View  Configuration 

To  achieve  near  constant  image 
plane  density,  the  modeler  can  employ 
some  of  the  level  of  scale  concepts  and 
add  smaller  objects  close  to  the  eye  at 
a  greater  density  to  provide  more  visual 
cues.  Continuing  with  the  example  above 
of  the  forest,  the  modeler  may  wish  to 
add  bushes  to  the  foreground  half  of  the 
image  plane  section  covered  by  trees, 
i.e.,  the  section  covered  by  angle  A  in 
Figure  2.  Suppose  that  the  modeler  has 
created  a  bush  which  has  a  transition 
range  to  be  brought  into  the  active 
scene  of  0.21  nm.  The  modeler  then  needs 
to  compete  the  necessary  density  of 

bushes  to  have  the  same  image  plane 
density  in  the  area  with  trees  and 

bushes  as  in  the  area  with  trees  only. 

The  density  of  trees  in  the 
120-degree  horizontal  field  of  view  was 
previously  computed  to  be  139  trees  per 
sqnm.  The  area  covered  Dy  the  annulus 
defined  by  angle  B  in  Figure  2  is  4.1 
sqnm.,  hence  825  trees  are  in  that 
section  of  the  field  of  view.  The 

region  defined  by  angle  A  in  Figure  2 
has  an  area  of  0.04  sqnm.  To  achieve 
the  s*me  density  of  objects  in  the 
section  of  the  image  plane  covered  by 
angle  A,  there  .must  also  be  825 
objects.  In  the  region  covered  by  angle 
A  there  are  presently  7  trees,  which 
means  that  the  modeler  needs  v.o  add  818 
bushes  to  achieve  the  desired  object 
density  in  the  image  plane.  818  bushes 


spread  out  over  the  area  covered  by 
angle  A  would  be  achieved  by  placing  a 
bush  on  the  average  of  every  40  ft. 

With  the  addition  of  the  bushes  to 
the  foreground,  the  modeler  has  achieved 
either  a  bush  or  a  tree  approximately 
every  40  ft.  as  compared  to  th  430  ft. 
with  only  the  trees.  If  the  modeler 

uses  4  surfaces  for  the  bush,  then  he 
will  have  achieved  a  surface  density  in 
the  forest  of  approximately  93,673 
surfaces  per  sqnm. 

The  93,673  surface  density 
represents  the  total  number  of  surfaces 
which  comprise  all  of  the  bushes  and 

trees  when  their  highest  level  of  detail 
is  presented.  Since  the  trees  have  a 
lower  level  of  detail  and  not  ali  the 

surfaces  in  the  bushes  and  trees  are 
front-faced,  only  about  4000  surfaces 
are  actually  displayed. 

Another  technique  for  achieving  a 
near  constant  image  plane  density  is  to 
have  the  same  type  of  objects  brought  in 
to  the  active  scene  at  varying 
distances.  In  the  forest  example  the 

modeler  may  choose  not  to  use  bushes, 
but  rather  have  the  trees  brought  into 
the  scane  at  several  transition  ranges. 
He  may  have  trees  coming  in  at  2  nm.  ,  1 

nm.  ,  and  0.5  nm.,  each  with  increasing 
densities  in  model  space  as  the  distance 
from  the  eye  decreases. 

By  trying  to  achieve  a  constant 
image  plane  density,  the  modeler  can 
create  a  data  base  which  has  an 
abundance  of  three-dimensional  cues  near 
the  eye  where  they  are  useful  instead  of 
concentrating  them  on  the  horizon.  These 
examples,  although  simplified,  show  the 
basic  principles.  In  practice  the 
modeler  can  calculate  the  densities  of 
several  sections  of  the  screen  and  use 
several  different  types  of  cDjecto  to 
achieve  the  desired  densities. 

Automated  Generation  Procedures 

In  the  creation  of  high  density 
data  bases,  the  modeler  is  faced  with 
the  problem  of  how  to  create  the  large 
number  of  objects  necessary.  In  the 
above  example  of  the  forest  with  bushes, 
if  the  modeler  were  to  create  each 
individual  bush  and  tree  in  a  50  nm.  x 
50  nm.  square  of  forert  he  would  have  to 
make  about  57  million  jushes  and  500,000 
trees.  The  need  for  automated  creation 
of  cbjects  is  clearly  evident.  It  is 
also  evident  that  the  image  generator 
must  be  able  to  reuse  portions  of  the 
data  base  since  it  cannot  hope  to  store 
the  number  of  surfaces  required  for  each 
individual  tree  and  bush. 

The  modeling  system  used  to  create 
CT5A  models  h«s  a  procedural  capability, 
which  gives  the  modeler  tremendous 
leverage  in  creating  large  numbers  of 

similar  objects.  These  procedures  are 


somewhat  similar  to  FORTRAN  subroutines 
which  have  input  parameters  and 
particular  outputs.  A  procedure  which 
makes  a  tree  can  have  input  parameters 
of  position,  scale,  rotation,  and 
transition  ranges  between  levels  of 
detail.  Once  the  procedure  is  written, 
it  can  be  called  many  times  with 
variations  to  the  input  parameters  to 
create  a  large  group  of  trees.  After 
this  group  of  trees  has  been  made,  it 
can  be  duplicated  and  used  in  many 
locations  to  create  a  whole  forest. 

The  creation  of  terrain  over  large 
geographical  areas  is  greatly  aided  by 
the  use  of  procedures.  Procedures  can 
be  written  to  create  mountainous  areas, 
hil]s,  farm  land,  forests,  swamps, 
oceans,  and  the  like.  If  the  modeler  is 
constrained  to  create  a  section  of 
terrain  to  a  certain  level  of 
specificity  he  may  do  so  and  then 
continue  the  enhancement  of  the  terrain 
by  the  use  of  procedures.  A  modeler  may 
specify  where  the  major  peaks,  valleys, 
and  ridges  are  in  a  mountainous  section 
and  then  let  the  procedures  embellish 
the  basic  structure  with  smaller  hills, 
minor  peaks,  and  vegetation. 

Procedures  defined  in  the  modeling 
system  may,  of  course,  call  other 
procedures.  This  capability  gives  even 
more  leverage  to  the  modeler.  A 
procedure  may  be  written  to  do  a  runway 
by  calling  other  procedures  to  do 
VASI's,  runway  lights,  strobes,  stripes, 
numbers-  oil  marks,  tire  marks,  and 
expansion  joint  cracks.  In  this  manner 
a  modeler  may  build  an  entire  runway  in 
only  a  few  minutes  by  specifying  such 
things  as  runway  heading,  length,  and 
type. 

The  largs  number  of  surfaces,  which 
must  be  stored  and  processed  in  a  high 
density  data  base,  also  requires  the 
image  generator  to  be  able  to  reuse 
portions  of  the  data  base.  The  CT5A  has 
an  instancing  capability  which  allows 
portions  of  a  data  base  to  be  stored  in 
memory  once,  but  used  many  times  bv 
relocation  within  the  data  base.  The 
modeler  uses  this  instancing  capability 
by  creating  a  mesh  for  a  particular 
portion  of  the  data  base  and  then 
indicating  that  copies  of  this  mesh 
should  be  placed  at  various  locations  in 
the  visual  environment.  The  original 
mesh  is  placed  in  the  image  generator's 
visual  environment  memory  and  then  all 
copies  access  the  same  information  fiom 
memory  and  apply  a  modeler-specified 
translation  to  position  the  mesh  where 
desired . 


entire  forest.  This  small  section  of 
forest  could  be  used  in  the  data  base  as 
needed.  In  this  manner  the  modeler  only 
has  to  mode?  a  small  section  of  the  data 
base  to  get  effectively  large  sections, 
while  freeing  the  image  generator's 
memory  to  hold  more  information 
concerning  otner  portions  of  the  visual 
envi ronment . 


EXAMPLES 

This  section  shows  several  examples 
of  data  bases  which  have  been  created  to 
demonstrate  the  modeling  strategies 
discussed  above.  These  data  bases  are 
all  designed  to  run  on  a  six-channel 
CT5A  with  a  120-degree  horizontal  field 
of  view  and  to  stay  within  the  system's 
nominal  capacity. 

Figure  3:  Hilly  forest  —  This 

hilly  forest  employs  a  single 
seven-surface  object  for  the  trees, 
using  fade  level  of  detail  for  the 
transition  from  null  to  a  single  tree. 
The  trees  are  brought  into  the  scene  at 
distances  ranging  from  1.5  nm.  to  0.5 
nm.  to  maintain  a  constant  image  plane 
density.  No  level  of  scale  concepts  are 
employed,  hence  the  lack  of  cues  for 
higher  altitudes  or  for  below  tree  top 
flying.  The  spacing  between  the  trees 
is  approximately  191  ft.,  creating  a 
surface  density  of  approximately  7,000 
surfaces  per  sqnm. 

Figure  4:  Flat  Forest  —  The  flat 
forest  employs  two  levels  of  detail  for 
the  trees  and  employs  fade  level  of 
detail  for  the  transitions.  Constant 
image  plane  density  is  achieved  by 
varying  the  distance  at  which  the  trees 
are  brought  into  the  scene.  The  trees 
and  the  abstract  ground  patterns  create 
two  levels  of  scale  and  enable  flight 
below  tree  top  level.  The  surface 
density  achieved  here  is  approximately 
130,000  surfaces  per  sqnm.  with  a 
spacing  between  the  trees  of 
approximately  50  ftet. 

Figure  5:  Ocean  and  Clouds 

Textural  features  like  ocean  waves  and 
clouds  are  possible  using  several  levels 
or  detail  and  several  levels  of  scale  on 
the  color  patterns.  The  procedures  used 
in  creating  the  ocean  and  clouds  are 
simila.  and  required  the  modeler  to 
specify  only  28  surfaces  for  the 
patterns.  The  ocean  and  clouds  have 
been  designed  to  use  only  34%  of  the 
system  channel  capacity,  thus  reserving 
sufficient  capacity  for  ships  and 
aircraft. 


When  creating  the  forest,  the 
modeler  may  create  a  group  of  575  bushes 
and  5  trees  as  a  mesh.  Using  the 
instancing  capability,  the  modeler  may 
then  place  this  group  of  bushes  and 
trees  at  various  locations  to  create 
portions  of  the  forest  or  perhaps  the 


Figure  6:  Desert  —  The  desert 

represents  the  most  thorough  application 
of  the  modeling  techniques.  Six  levels 
of  scale  are  used  to  create  a  nearly 

constant  image  plane  density  and  to 

support  flight  at  a  broad  range  of 

altitudes.  Each  item  in  the  desert  is 
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modeled  at  2  to  4  levels  of  detail, 
using  fade  level  of  detail  for  the 
transitions.  The  surface  density 

achieved  is  approximately  87,000 
surfaces  per  sqnm.  over  100  sqnm.  and 
yet,  using  the  procedural  capabilities, 
the  modeler  only  had  to  model 
approximately  400  surfaces.  Using  the 
hardware  instancing,  the  entire  desert 
requires  only  4000  suifaces  in  the  image 
generator's  memory. 


CONCLUSIONS 

An  effective  combination  of  the 
CT5A  hardware  capabilities  and  modeling 
strategies  enables  the  modeler  to  create 
and  display  data  bases  which  greatly 
increase  the  usefulness  of  the  CIG 
image.  The  image  generator's  ability  to 
process  large  amounts  of  information, 
and  make  a  multitude  of  decisions 
concerning  such  things  as  level  of 
detail,  load  management,  perspective 
foreshortening  correction,  and  data  base 
culling  in  real  time,  enables  the 
modeling  strategies  to  be  employed. 

It  is  our  belief  that  by  applying 
these  modeling  techniques,  along  with 
the  necessary  hardware  support,  visual 
data  bases  can  be  created  which  will 
support  nap  of  the  earth,  contour 
flying,  vertical  takeoff  and  landing, 
and  a  host  of  other  applications. 
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ABSTRACT 


Constructing  a  scene  data  base  for  current  computer  image  generation  systems  is  a 
costly  ar.d  time-consuming  task.  Thousands  of  edges  must  be  defined  by  positioning  the  end 
points,  or  vertices,  of  each  edge.  In  addition,  edges  bounding  a  common  surface  or  face 
must  be  linked  in  a  list.  Data  for  each  face  must  include  information  for  a  normal  vector, 
and  data  for  faces  representing  curved  objects  must  include  information  for  normal  interpol¬ 
ation  to  simulate  smooth  shadinq  across  the  object.  This  paper  describes  a  more  efficient 
scene  model  that  is  easier  to  construct  and  yet  produces  a  more  faithful  representation  of 
the  real  world.  Scene  geometry  is  modeled  by  quadric  surfaces  bounded  by  planes.  Scene 
detail  is  modeled  by  a  mathematical  texturinq  function  which  modulates  surface  shading 
intensity  and  translucence.  The  paper  describes  how  the  new  model  simplifies  modeling 
terrain,  cultural  features,  moving  targets,  and  special  effects^ 


INTRODUCTION 

The  problan  of  constructing  a  computer  model 
representing  the  complexity  of  a  real-world  scene 
and  allowinq  real-time  image  generation  is  a 
formidable  one.  Two  decades  ago,  corvuter  image 
generation  (CIG)  pioneers  attacked  this  problem  by 
apply inq  a  time-honored  engineering  axiom:  "be  wise 
-  linearize."  The  linear  scene  moud  that  they 
prrducea  was  defined  by  efficiency  of  imaqe 
computation  rather  than  by  efficiency  of 
model  inq.  Over  the  past  20  ye.^s,  impressive 
developments  in  CIG  techniques  based  on  this  model 
have  reinforced  its  use.  Over  the  same  period  of 
time,  however,  the  sophistication  and  complexity  of 
training  missions  have  created  new  demands  on  CIG 
technology.  Ironically,  it  is  the  past  successes 
of  CIG  that  have-  enabled  traininq  to  move  forward 
to  the  point  where  it  can  make  these  demands. 

1  urrent  traininq  requirements  call  for  the 
capability  to  model  extensive  qaminq  areas  and  to 
represent  scene  detail  with  enough  fidelity  to 
support  nap-of-the-earth  (NOE)  flight  and  qround 
based  operations.  The  asymptotic  progress  of  edqe 
CIG  technology  indicates  that  these  requi rements 
can  be  satisfied  only  by  a  new,  more  efficient 
approach  to  scene  modeling. 

A  NEW  SCENE  MODEL 

In  developing  a  new  scene  model,  Grumman 
studied  the  problem  from  the  top  down,  choosinq  the 
most  efficient  data  base  to  represent  the  full 
range  of  scene  features  needed  for  today's 
traininq.  These  features  include  terrain, 

vegetation,  cultural  features,  moving  targets, 
clouds,  smoke,  and  weapons  effects.  We  determined 
that  the  most  significant  limitations  in  edge 
technology  for  nodell nq  these  features  were  due  to 
its  inability  to  model  surface  curvature  and 
textural  detail  efficiently.  We  therefore  focused 
our  scene  model  development  on  devising  efficient. 
representation  of  these  two  features. 

Surface  Curvature 


Ouadric  surfaces 
simplest  fornn  of  curved 


are  matnemacical ly  the 
surface  and  can  represent 


individual  scene  features  as  individual  volumes. 
This  allows  modelinq  many  scene  features,  such  as  a 
bush  or  a  toulder,  by  a  single  ellipsoid.  Modeling 
such  features  in  this  way  is  much  simpler  than 
using  edges  because  very  few  parameters  are 
required  to  define  a  quadric  surface,  and  these 
parameters  relate  directly  to  the  shape  ard 

position  of  the  feature  beinq  modeled. 

Many  scene  features  can  not  be  represented  by 
a  sinqle  quadric  surface  alone.  T^  allow  more 
flexibility  in  our  scene  model,  we  include  the 

capability,  to  bound  a  quadric  surface  with  planar 
surfaces^1 '.  This  allows  us  to  model  a  wide 
variety  of  curved  and  linear  scene  features  in  a 
very  simple  manner.  For  example.  i’*e  can  model  an 

aerodynamic  surface  usinq  a  thin  elliptical  cone 

bounded  by  two  planes.  A  qun  barrel  or  tree  trunk 
can  be  modeled  by  a  cylinder  bounded  by  two 
planes.  We  orqanize  our  data  base  as  a  set  of  such 
geometrical ly  defined  "objects,"  with  each  object 
defined  by  one  quadric  surface  and  up  to  six 

bounding  planes.  Complex  scene  features,  such  as 
targets,  are  modeled  by  nultiple  objects.  A  qlance 
at  Fig.  1  will  show  how  this  pa-ametric  approach 
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F»9.  1  Compomon  of  Two  Tr**  Models 


simplifies  scene  modeling  compa  .  ed  to  edqe 

modelinq.  Usinq  edqes,  the  foliage  for  a  single 
tree  would  require  the  positioning  of  many 
vertices,  with  lists  linking  vertices,  edqes,  and 
faces.  iJsinq  quadric  surfaces,  we  need  specify 
only  three  dimension  parameters,  three  rotation 
anqles,  and  three  position  coordinates  for  an 
ellipsoid,  in  addition  to  three  rotation  anqles  and 
three  position  coordinates  for  a  houndinq  plane. 
The  tree  trunk  can  he  modeled  similarly  by  a 
cylinder  and  two  bounding  planes.  Thus,  the 

modeler  will  be  required  to  define  fewer  data 
values,  and  these  values  will  ha''e  a  direct 
relation  to  the  qeonetry  of  scene  features  he  is 
model  i  nq. 

The  significance  of  this  simplification  lies 
in  the  fact  that  any  realistic  scene  will  contain 
hundreds  or  even  thousands  of  such  features. 
Automated  as  well  as  manual  scene  modeling  will 

benefit  from  this  simplification  since  less  data  is 
involved,  and  the  parametric  nature  of  the  data 

base  allows  straightforward  definition  of  shape, 
size,  and  position. 

Textural  Detail 

Most  of  the  visual  detail  in  real  world  scenes 
is  due  to  minor  topcqraphi cal  variations  that  would 
be  very  costly  to  model  with  a  geometric  data 
base.  Such  detail  can  he  efficiently  modeled  usinq 
texture  patterns  mapped  to  the  appropriate 
qeometric  surfaces.  In  studying  various  approaches 
to  texturinq,  we  developed  a  mathematical  function 
that  modulates  surface  shadinq  intensity  as  the 
imaqe  is  qenerated^.  The  texture  function  maps 
the  pattern  directly  to  scene  surfaces  with  true 
perspective  validity  because  its  independent 
variables  are  the  scene  coordinates.  This  approach 
produces  a  very  compact  data  base  because  only  25 
function  parameters  define  complex  patterns 
covering  larqe  scene  areas.  In  addition,  many 
scene  features  can  be  textured  usinq  the  same  set 
of  parameters.  Because  the  pattern  depends  on  the 
scene  coordinates,  each  feature  will  look  different 
fron  all  the  others.  In  this  way  we  can  model  an 
unlimited  number  of  individual  trees  or  hills  using 
a  sinqle  set  of  texturing  function  parameters. 
Because  the  function  parameters  can  be  related  to 
the  spectral  content  of  the  texture  pattern, 
modelinq  can  be  based  on  Fourier  analyses  of  images 
of  real-world  features.  Thus,  the  texture  function 
has  the  potential  for  automated  modeling. 

One  of  the  qreatest  advantages  of  the 
texturinq  function  technique  is  the  ability  to 
control  the  function  parameters  on-line  during 

image  generation.  This  permits  eliminating  any 
hiqh- frequency  content  of  the  pattern  that  would 
cause  aliasinq.  Related  to  this  feature  is  the 
ability  to  vary  the  detail  of  the  pattern  as  a 
function  of  range.  This  provides  a  very  efficient 
means  of  varying  1 evel-of-detai  1  (LOD)  without 
chanqinq  the  geometric  data  base.  Control  of  the 

function  parameters  also  allows  us  to  vary 
translucence  across  the  surface  of  an  object  to 
enhance  the  modelinq  capability  by  simulating 

irreqular  boundaries  and  holes.  Finally,  by 
varyinq  the  parameters  dynamically,  we  can  simulate 
motion,  such  as  tree  leaf  agitation  and  smoke 

risinq. 

In  summary,  the  texture  function  simplifies 
modelinq  of  scene  detail  by  employing  a  minimi 


data  base  that  can  be  related  directly  to  the 
visual  characleri sties  of  specific  real-world 
features.  In  addition,  the  texture  function  allows 
an  efficient  means  of  modeling  irregular  features, 
such  as  trees,  clouds,  and  smoke,  with  dynamic 
capabi 1 itv. 

SCENE  MODELING  WITH  TEXTURED,  QUADRIC  SURFACES 

Although  quadric  surfaces  have  been  recognized 
as  potentially  useful  for  modeling  a  limited  set  of 
cultural  features'3',  they  have  generally  been 
considered  too  simple  to  model  the  wide  range  of 
features  required  for  combat  training'4'.  We  have 
found  that  the  limitations  of  quadric  surfaces  are 
qreatl.y  minimized  by  the  addition  of  texturing,  and 
that  the  combination  of  quadric  surfaces  and 
texturinq  provides  an  efficient  data  base  for 
modeling  the  full  ranqe  of  features  required  for 
traininq,  including  cultural  featuras,  terrain,  and 
special  features. 

Cultural  Features 

Many  cultural  features  can  be  modeled  by  a 
sinqle  bounded  quadric  surface.  Cylinders  can  be 
used  for  oil  tanks  and  poles;  spheres  can  be  used 
fo  fuel  tanks,  lights  of  finite  size,  and  radar 
domes.  More  complex  cultural  features  can  be 
modeled  by  sets  of  objects  each  consisting  of  a 
bounded  quadric  surface.  Thus  we  can  use  a  single 
object  to  model  a  w,nq,  a  canopy,  an  engine  pod,  a 
refuelinq  boom,  a  tank  turret,  a  gun  barrel,  a 
smoke  stack,  a  wheel,  or  even  a  helicopter  rotor 
disk.  Complicated  curved  bodies,  such  as  an 
aircraft  fuselage,  can  be  either  approximated  by  a 
single  object  or  modeled  more  accurately  by 
multiple  objects.  In  the  latter  case,  care  must  be 
taken  to  fit  abutting  objects  without  noticeable 
surface  discontinuities.  The  simplicity  of  the 
mathematics  of  quadric  surfaces  provides  the 
potential  for  automated  modeling  aids  to  alleviate 
this  problem.  The  simplicity  of  modelinq  cultural 
features  typical  of  training  scenes  is  demonstrated 
in  Fi  q.  2,  which  shows  how  a  tank  can  be  modeled 
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efficiently  with  only 
a  high  LOD  version  of 


five  objects.  Figure  3  shows 
the  tank  using  51  objects. 
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Fig.  3  High  Level -ot-Oetail  Tank  Model  Composed  of  51  Objects 


Because  each  of  our  objects  can  include  up  to 
six  hounding  planes,  they  can  be  used  to  model 
linear,  as  well  as  curved,  cultural  features.  To 
model  complex  linear  features  such  as  factories,  we 
developed  a  program  called  "BLOCK,"  which  creates 
buildinqs  of  arbitrary  complexity  using  simple 
linear  objects  as  building  blocks.  The  building 
blocks  are  rectanqular  solids  simply  def.ried  ny 
three  position  and  three  dimension  parameters. 


Our  texture  function  can  be  used  to  add  detail 
to  a  culcural  feature  model.  This  detail  can  be 
linear,  as  in  stripes  on  building  walls,  or  non¬ 
linear,  as  in  camouflage  markings  on  targets. 

An  important  consideration  in  modeling  larqe 
qaminq  are-s  is  implementation  of  the  Defense 
Mappinq  Aqency  (DMA)  cultural  data  information. 
Because  this  information  defines  features  in 
qeneric  terms  only,  it  will  be  necessary  to 
construct  a  library  of  qeneric  cultural  features 
which  can  be  accessed,  scaled,  and  positioned  in 
the  scene  in  accordance  with  the  DMA  data.  The 
efficiency  of  cur  object  modeling  will  facilitate 
the  construction  of  such  a  library,  and  the 
parametric  nature  of  the  model inq  will  simplify 
scalinq  and  positioning  the  features. 

Terrain 


The  task  of  modelinq  arbitrary  terrain  has 
generally  been  approached  by  treating  an  extensive 
reqion  as  a  sinqle,  complicated  surface  that  can  be 
approximated  by  a  larqe  number  of  planar  patches. 
Because  a  linear  model  is  so  inefficient  for 
modeling  complex  curved  surface*.  an  unaccepiut le 
number  of  edges  would  be  required  for  a  realistic 
representa* i on  As  a  result,  real-time  edge 
svstens  rust  limit  the  edqe  allocation  and  produce 
primitive  terrain  imaoes  dominated  by  sharp  linear 
boundaries. 


A  more  e.ficient  .vid  effective  way  to  model 
terrain  is  to  represent  it  as  the  eye  perceives  it, 
as  a  set  of  individual  topoqraphica!  features,  such 
as  hilis,  mountain  peaks,  and  ridqes.  Quadric 
surfaces  lend  themselves  to  this  approach  because 
each  major  feature  can  be  modeled  by  a  sinqle 
curved  surface  free  of  linear  artifacts.  The 


quadric  surface  can  be  defined  by  10  parameter 
specifying  its  shape  ana  orientation  in  the 
scene.  Although  isolated  quadric  surface  objects 
can  be  used  to  model  some  terrain  features,  in 
general,  clusters  of  abutting  objects  will  be 
required  to  provide  a  satisfactory  representation 
of  arbitrary  terrain.  This  requires  determining 
appropriate  bounding  planes  to  provide  continuity 
between  adjacent  quadric  surfaces.  Figure  4  shows 
schematical ly  how  sinqle  or  multiple  quadric 
surface  objects  can  be  used  to  model  terrain 
features. 


ISC  LATE  D  OBJECT 


GROJND  PLAME 


SOUND'NG  PLANE  BOUNDING  PLANE 


HOHIJONTALLV  ABUTTING  OBJECTS 


VER  T ICAl.LV  ABUTTING  OBJECTS 


1 199-Ooru 

Fig.  4  Examples  of  Terrain  Features  Modeled  by  Single  and  Multiple 
Bounded  Uuadnc  Surface  Objects 

Modeling  the  significant  terrain  features  with 
quadric  surfaces  will  produce  a  very  compact  data 
base  free  of  linear  artifacts,  but  will  not  by 
itself  provide  enough  scene  content  for  traininq. 
Our  texturinq  function  will  produce  the  textural 
detail  required  to  model  thp  secondary  topographi¬ 
cal  detail  common  in  real-world  terrain.  A  single 
texture  pattern,  defined  by  one  set  of  texture 
function  parameters,  ran  be  used  to  texture  all 
surfaces,  including  the  ground  plane,  within  a 
lefinei  region  of  arbitrary  size.  This  provides  a 
great  leal  of  scene  content  with  a  minimal  data 
base.  In  addition,  the  te-cturinq  enhances  the 

appeanance  of  surface  continuity  in  the  scene. 
Because  wn  model  majon  tenrain  features 
individually,  we  can  also  ass*qn  different  texture 
patterns  to  different  features.  This  w’  1 1  allow  us 
to  represent  qrass-covered  hills,  rouky  mountains, 
ani  forest  areas  in  the  scene. 


To  satisfy  the  requirements  ot  training 
missions  involvinq  low-level  flight  and  ground- 
based  operations,  we  rust  be  able  :o  model  ground 
features,  such  as  tree*  arid  rocks,  with  a  fair 
amount  of  realism.  Trees  have  been  a  particular 
stunblinq  block  in  edqe  modelinq  because  edqes 
define  features  n  a  very  explicit  manner.  Thus, 
hundreds  or  even  thousands  of  edges  would  be 
required  Ij  represent  the  complexity  of  a  tree's 
foliage.  Using  quadric  surfaces,  we  can  model  a 
tree  with  ore  or  two  textured  abjects.  The 
parametric  control  inherent  *n  our  texturing 
function  provides  the  irregular  surface  an.J 
boundary  detail  necessary  to  represent  the  essence 
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of  tree  foliaqe  without  explicitly  modeling  each 
branch  and  leaf.  An  added  advantage  to  this 
approach  is  the  fact  that  the  texture  intensity  is 
added  to  the  quadric  surface  shading  intensity.  As 
a  result,  the  overall  shading  of  the  model  is 
representati ve  of  the  illumination  source,  with 
"leaves"  on  the  underside  darker  than  thos°  on 
tOD.  In  a  similar  manner,  rocks  and  boulders  can 
be  modeled  accurately  by  a  single  textured  quadric 
surface  for  each. 

To  take  advantaqe  of  quadric  surface  model inq 
of  terrain  features,  we  developed  a  computer 
proqram  called  "CNGEN"  which  qenerates  clusters  of 
hills,  trees,  and  rocks.  CNGEN  operates  on  a  very 
corpact  list  of  feature  clusters.  The  data  for 
each  cluster  include: 

(1)  coordinates  and  dimensions  of  a 
rectanqular  reqion  on  the  scene  ground 
plane  and  a  qrid  spacinq  for  positioning 
individual  cluster  features  in  the 
reqion, 

(2)  shape,  and  relative  position  parameters 
for  template  objects  representing  a 
typical  cluster*  feature, 

(3)  an  index  to  a  list  of  texture  function 
parameter  sets, 

(4)  a  set  of  three  color  parameters  (RGB), 
and 

(5)  a  translucence  value. 

For  each  cluster  in  the  list,  CNGEN  adds  pseudo¬ 
random  increments  to  the  qrid  position,  color,  and 
shape  parameters  of  each  individual  feature  and 
stores  the  features  in  a  scene  data  base  list. 
CNGEN  greatly  simplifies  creating  a  ♦e'rain  model 
of  a  large  qaminq  area,  and  editinq  the  .cene  is 
facilitated  by  the  capability  to  change  the 
position,  extent,  or  character  of  a  iaroe  number  of 
objects  at  cnce.  Different  texture  patterns  or 
colors  can  be  specified  as  well  as  different  shapes 
and  -  i 2<?s  for  cluster  features.  Because  of  the 
efficiency  of  quadric  surface  model inq,  the 
confutation  involved  in  CNGEN  is  minimal, 
suqqestinq  the  feasib<lity  of  on-line  scene 
generation. 

Tne  simplicity  and  effectiveness  of  textured 
quadric  surface  mode* inq  ure  demonstrated  in  Fig.  5 
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Fif.  S  Scant  Composed  of  173  Beundwf  Quod**  Surfac* 

Obyocn  Without  Tsaturn 


and  6.  Figure  5  shows  a  terrain  scene  composed  of 
quadric  surfaces  without  texturing.  Fiqure  6  shows 
the  same  scene  with  texturinq.  This  model  was 
generated  by  CNGEN  from  a  list  of  18  feature 
clusters  defined  by  a  total  of  400  data  values. 
The  texture  parameter  file  included  data  for  seven 
texture  patterns  defined  by  a  total  of  175  data 
values.  The  final  model  included  173  objects,  many 
of  which  are  not  visible  in  the  image.  We 
generated  a  dynamic  sequence  of  NOE  flight  through 
this  scene  wnich  confirmed  the  perspective  validity 
of  the  model inq  approach  and  demonstrated  excellent 
f lying  cues  produced  by  the  high  scene  content. 
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Fig.  6  Tern>n  Scvrw  Compowd  of  173  Bounded  Quadric  Surface 
Object*  with  Scene  Detail  Produced  by  Texturing  Function 


To  provide  full  terrain  modeling  capability 
for  today's  training  requirements,  we  will  have  to 
use  t*e  DMA  elevation  data  as  a  primary  source.  We 
have  performed  preliminary  research  in  this  area 
and  have  developed  an  approach  to  modeling  the  DMA 
data  with  teutured  quadric  surfaces.  This  approach 
includes  the  following  steps: 

(1)  Determine  major  te;r,»in  features  by  low 
pass  digital  filtering  of  the  DMA 
elevation  data. 

(2)  Isolate  major  features  in  the  filtered 
data  using  digital  image  processing  and 
pattern  recognition  techniques. 

(3)  Fit  a  single  quadric  surface  to  the 
o»  ieir.al  DMA  data  corresponding  to  each 
itilated  major  feature. 

(4)  Determine  appropriate  bounding  planes  for 
earn  feature  to  K-axinize  continuity 
between  adjoining  surfaces. 

(5)  Determine  appropriate  texture  function 
parameters  by  Fourier  ana’ysi*  of  the 
original  OKA  data. 

Under  a  National  Science  Foundation  (NSF )  grant 
investigated  the  feasibility  of  our  approach  b, 
means  of  a  t**o-di»ens fona’  analysis  in  wMch  we 
filtered  profiles  of  actual  digitized  elevation 
data.  Isolated  peaks,  and  fit  conic  section 
curves'  .  The  success  of  this  wort  provides  a 
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firm  basis  for  extension  of  the  techniques  to  three 
dimensions.  Terrain  models  generated  in  this 
fashion  w\  1 1  produce  a  realistic,  nonlinear  data 
base  that  will  qreatly  compress  the  in.ormation 
contained  in  the  DMA  elevation  files.  Our  CNGEN 
techniques  will  then  be  extended  to  ennance  the 
terrain  elevation  model  by  addinq  trees,  rocks,  and 
other  surface  objects. 

Special  Features 

Althouqh  the  qreat  majority  of  scene  objects 
can  be  modeled  as  solid  objects  with  clearly 
defined  boundaries,  today's  military  traininq 
scenarios  include  scene  features,  such  as  weapons 
blasts,  smoke,  dust,  and  clouds,  which  can  not  be 
represented  in  the  same  explicit  manner  and  must  be 
treated  as  special  features.  These  features  are 
characteri  zed  by  a  lack  of  solidity  and  by 

amorphous  shapes  with  irreqular  and  poorly  defined 
boundaries.  In  addition,  they  are  qenerally 
dynamic,  with  constantly  changinq  structure. 
Although  these  features  are  serious  stumbling 
blocks  for  edqe  modelinq,  they  are  very  efficiently 
modeled  by  textured  quadric  surfaces.  Just  as  we 
modeled  the  foliaqe  of  a  tree  usinq  a  sioqle 

textured  quadric  surface  with  variable 
translucence,  so  we  can  model  a  col  mn  ot  smoke,  an 

atmospheric  cloud,  a  dust  cloud,  or  a  weapon 

burst.  Control  of  the  texture  function  parameters 
allows  the  variable  translucence  required  to 
simulate  an  amorphous  shape.  In  addition,  we  can 
vary  the  parameters  from  frame  to  frame  to  aqitate 
the  texture  pattern  to  simulate  motic...  The 
simplicity  of  quadric  surface  shape  definition  even 
a  1 1  cms  us  to  create  expandinq  weapons  bursts. 

Some  of  the  possible  special  features  that 
q  adric  surfaces  can  model  effectively  are  shown  1r. 
Fiq.  7  and  8.  Fiqure  7  shows  smoke  from  chimneys 
on  a  factory.  The  clouds  in  the  sky  are  modeled  by 
mapping  texture  onto  a  plane  in  the  sky.  Fiqure  9 
shows  *anks  with  dust  clouds.  We  generated  a 
dynamic  sequence  of  this  scene  which  demonstrated 
the  realism  of  the  motion  of  the  dust.  The  effect 
was  produced  by  moving  t*e  quadric  surfaces 
assiqned  to  the  Oust  clouds  with  the  tanks,  while 
movinq  the  texture  pattern  on  the  dust  objects  at  a 
slower  speed.  Figures  7  and  8  also  demonstrate  the 
modeling  of  textured  linear  features  (the  factory), 
textured  targets  vthe  tanks),  rolling  terrain, 
rocks,  and  bushes. 

ADDITIONAL  ADVANTAGES  OF  THE  NEW  SCENE  MODfl 


The  sophistication  of  nod ern  training 
scenarios  places  stringent  demands  on  scene 
modeling  for  CIG  systems.  Viewable  sc«»ne  content 
must  be  carefully  managed  to  prevent  system 
overloads  due  to  excessive  computation  leads  during 
Image  generation.  Specific  enemy  targets  rust  be 
nodelee  with  enough  fidelity  to  allow  recognition 
at  reasonabl?  distances,  and  provision  must  be  made 
to  allow  tagets  to  wove  arbitrarily  in  the  sce^e. 


Scene  Htnagement 

Hpdeling  scent  features  in  auUiple 
detail  has  proven  to  be  an  effective 
controlling  scene  content.  But  this 
impacts  the  scene  modeling  task  by 
features  to  be  modeled  several  times, 
with  textured  q  .adric  surfaces 
implementation  bi  ;QD  sriydelirg 


levels  of 
means  of 
technique 
requiring 
Hodeling 
simplifies  the 
in  tnree  ways. 
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Quaoric  Surfaat  Obpcts  md  Tunning  Function 


First,  nany  scene  features  can  be  modeled  in  a 
single  LOO  using  a  sinqle  textured  quadric 
surface.  Control  of  the  texture  function  to 
eliminate  high  frequency  detail  that  would  produce 
aliasing  automatical  ly  provides  LOD  management. 
Thus  the  same  model  can  be  used  reqardless  of  the 
viewing  ranqe.  Second,  large  clusters  of 
individual  scene  features,  such  as  groups  of  trees, 
car.  he  modeled  in  just  two  levels  of  detail,  a  high 
LOD  m/del  consisting  of  objects  representing  the 
individual  features  and  a  low  LOD  model  consisting 
of  one  object  representing  the  cluster.  By 
inserting  a  moveable  bounding  plane  at  a  fi?:ed 
distance  from  the  viewpoint,  we  can  continuourly 
truncate  tho  low  LOD  object  as  the  viewpoint 
approaches  while  smoothly  Inserting  the  high  LOD 
objects  in  front  of  (be  bounding  plane.  As  shown 
in  Fig.  9,  this  technique  will  be  effective  for  low 
altitude  approaches  that  would  '.ot  lend  themselves 
to  a  sirple  fadinq  of  the  ow  LOO  object  by 
increasing  its  translucence.  This  technique  has 
the  added  advantage  that  translucence  need  be 
applied  only  to  the  high  LOO  objects  near  the 
bounding  plane  to  smooth  the  LOO  transition.  In 
adviMon,  texturing  will  jreatly  enhance  the 
effectiveness  of  both  LOO  models.  Final iy,  the 
slmpl if ication  in  nodeliig  yrovided  by  textured 
quadric  surfaces  will  apply  to  all  levels  of 
detai\  Thus  even  features  requiring  more  than  two 
levels  of  detail  will  be  easier  to  represent. 


1 19*00*0 


Fig.  8  Gamoofl»9*d  Tanks  with  Oust  C>>uds  in  Rolling  Terrciti 
Msdel«d  by  Bounded  Quadric  Surfaeas  and  TeHturtng 
Function 

Target  Model  inq_ 

In  order  to  provide  a  trainee  with  the 
capability  for  tarqet  recognition  in  a  cluttered 
..aminq  area,  we  must  represent  all  potential 
targets  with  a  reasonable  amount  of  fidelity.  This 
includes  friendly  as  well  as  enemy  vehicles, 
dbdeiinq  such  vehicles  with  *»dqes  requires  the 
positioning  of  many  vertices  using  a  physical  model 
or  engineering  drawings.  Quadric  surfaces  can 
simplify  model ir  i  because  they  are  closely  related 
to  the  mathematical  information  used  in  engineering 
desiqn.  Engineering  line  drawings  are  generally 
produces  from  cross-sections,  or  stations,  defined 
alonq  a  lonqitudinal  axis.  pach  station  is  defined 
in  terras  of  conic  sections,  and  surfaces  between 
Stations  are  defined  by  second-order  variations  in 
the  conic  sections.  Conic  sections  are  identical 
to  sections  of  intersection  between  quadric 
surfaces  and  p^nes,  and  quadric  surfaces  have 
second-order  variation.  Thus,  a  modeler  can  use 
this  information  to  define  quadric  surfaces,  and 
the  potential  exists  for  automated  construction  of 
quadric  models  us<nq  standard  engineering  design 
data. 


Moving  Targets 

Moielinq  a  moving  tarqet  is  no  different  than 
modeling  a  stationary  scene  feature.  It  must  be 
kept  ii  mind,  however,  that  any  model  data 
parameters  defininq  position  will  have  to  be 

chanqed  on-line  to  reposition  the  model  for  each 
imaqe  frame.  In  edqe  model  inq,  all  data 
parameters,  includinq  vertex  coordinates  and 
surface  normals,  relate  to  position.  In  quadric 
surface  modeling,  the  shape  parameters  are  not. 
position  dependent  and  therefore  need  not  be 
chanqed.  m  addition,  quadric  surface  modeling 
produces  a  m-ji  more  compact  data  base  than  edqe 
model inq,  so  fewer  surfaces  will  Have  to  be 

moved.  Thi *  ic  part i cul  arly  important  in 

ill)  lenient  inq  movl.iy  models  with  novinq  parts,  where 
nuUiple  t ransf^rmatl  ons  of  coordinates  are 
i nvol ved. 

Ground  following  tarqets  present  a 
particularly  difficult  problem  for  real-time  C!G. 
The  biqqest  part  of  this  problem  is  to  determine  on 
which  surface  a  tarqet  lies,  so  that  its 

orientation  can  be  adjusted  to  fit  the  surface 

slope.  Quadric  surface  modelinq  alleviates  this 
problem  because  it  produces  fewer  terrain  surfaces 
upon  which  a  tarqet  can  lie.  In  addition, 

structuring  the  terrain  model  us  a  sec  of 

individual  objects  allows  tne  capability  of  testinq 
ill  ooiects  in  parallel  for  the  presence  of  a 
target. 

i  IT'JRE  EFFORT 

We  have  demonstrated  how  textured  quadric 

surfaces  will  solve  nan y  scene  model inq  problems. 

Hcwever,  we  have  only  discussed  how  this  approach 
can  be  applied  to  other  scene  modeling  problems. 
The  next  task  will  be  to  test  and  refine  our 
solutions  to  these  remaining  prohlems. 

The  nest  important  problem  remaining  is  to 
develop  techniques  tc  automat ical ly  model  the  UNA 
elevation  data  with  textured  quadric  surfaces.  To 
do  this  we  will  extend  our  profile  analysis 

technique,  to  -  >ree  dimensions.  Next  we  will 
extend  the  scene  modelinq  techn4rUes  usel  in  our 
CNGEN  program  to  place  trees  and  rocks  o m  terrain 
surfaces  other  than  the  ground  plane. 


We  will  also  develop  modeling  aids  to 
facilitate  the  nodelinq  of  cultural  features,  such 
as  tarqets.  We  will  use  these  aids  to  construct  a 
library  to  be  used  in  conjunction  with  the  DMA 
cultural  file  to  supplement  the  terrain  elevation 
data. 

It  has  become  clear  to  us  that  efficient  scene 
modeling  will  require  the  inclusion  of  twu- 
dimensional  feature  models.  We  plan  to  adapt  our 
basic  bounded  quadric  surface  model  to  two 
dimensions  in  the  form  of  I  '■  nearly-bounded 
curves.  We  will  then  be  able  to  model  and  image 
such  features  as  roads,  rivers,  and  lakes  more 
simply. 

We  will  develop  st  our  LOD  management 

techniques,  and,  final  «e  will  investigate  the 
feasibility  of  our  CNGEN  and  BLOCK  techniques  for 
on-line  generation  of  scene  features. 

CONCLUSIONS 

We  have  described  a  new  scene  model,  cc-.ifosed 
of  bounded  quadric  surfaces  and  texturing,  which 
simplifies  scene  generation  compared  to  edqe 
techniques.  The  new  scene  model  is  defined  by  a 

parametric  data  base  which  is  directly  related  to 
the  size,  shape,  and  petition  of  scene  features. 

In  addition,  the  new  scene  model  produces  a  much 

more  compact  data  base  because  fewer  surfaces  can 
be  used  to  model  nonlinear  features,  common  in  the 
real  world.  The  effectiveness  of  the  new  model  is 

underlined  by  its  capability  to  mode!  efficiently 
such  irregular  features  as  trees,  smoke,  and 
clouds,  whicn  are  serious  stumbling  blocks  for  edqe 
model inq. 
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This  paper  will  present  programming  techniques  and  mathematical  algorithms  for  produci’1'’ 
animated  sequences  of  exploding  objects  such  as  buildings  and  ship  targets.  The  visial  effect 
has  obvious  extension  for  application  in  simulation  of  combat  conditions.  Potential  applications 
include  simulation  of  weapons  effects  on  real-time  CIG  visual  systems. 
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INTRODUCTION 

Computer  animation  continues  to  be  a  valuable 
tool  lor  the  algorithmic  generation  oi 
time-dependent  phenomena.  Techniques  lot  the 
display  it  motion  that  have  been  developed  over 
the  past  13  years,  combined  with  display 
algorithms  for  high  resolution  imagery,  enables 
one  to  generote  more  realistic  pictures  and 
animation.  These  displays  can  add  to  our 
understanding  of  complex  processes. 

In  our  r 'search  we  concentrated  on  developing 
tools  tor  Highly  detailed  data  base  generation. 

The  emphasis  has  been  upon  interactive  computer 
graphics  techniques  to  work  with  a  Defense  Mapping 
Agency  data  bc.se  adding  cultural  features.  We 
also  generated  at  non-real  time  rates  animation 
which  simulated  moving  along  a  shoreline  oi 
Norfolk,  Virginia.  During  the  course  ot  this 
effort  we  have  considered  the  requirements  for 
real-time  d..,play  and  the  kinds  of  visual  cuts  one 
might  create  to  make  the  simulators  more 
realistic.  It  occured  to  us,  perhaps 
serend ipidously ,  that  we  might  be  :ble  to  i leu.  ,te 
explosions  because  u!  out  delta  structures  aud 
display  algorithm. 

We  have  developed  a  program  that  we  believe 
has  appl icat ions  tor  simulators  and  training.  The 
realism  ot  combat  Situations  portrayed  in  a 
simulator  with  graphics  capabilities  could  be 
enhanced  if,  along  with  maneuvers  oi  t he  craft  or 
vessels  engaged,  the  destruction  of  targets  were 
simulated  as  well.  The  program  takes  an  object 
described  as  three-dimensional  data  and  animates 
the  inoividuai  polygons  that  define  its  shape. 

This  technique  has  been  used  successfully  to 
"explode"  and  "implode”  objects  and  to  alter  the 
shapi  of  objects.  For  example*,  t he  first 
implementation  of  the  program  was  the  explosion  of 
a  realistic  model  of  a  tank.  It  should  be  noted 
early  in  this  piper  that  the  goal  of  this  program 
is  not  to  create  lifelike  explosions.  The 
explosion  of  a  physical  object  is  an 
extraordinarily  complex  phenomenon  governed  by 
many  faccors,  including  the  nsture  of  the 
explosive  and  the  manner  of  its  detona'iou,  and 
the  properties  of  the  exploding  object  and  its 
surroundings.  These  factors  affect  the  size  of 
the  blast  and  the  amount  of  smoke,  flame,  dust  and 
debris  produced.  For  our  purposes  we  are  not 


interested  in  faithfully  reproducing  the  processes 
it. a*  constitute  such  an  event.  Rather,  we  wish  to 
produce  a  visually  interesting  and  unambiguous 
suggestion  of  a  detonation. 

Ine  program  is  general  enough  to  al.iow  •or 
numerous  applications.  Buildings,  bridges,  towers 
and  structures  of  any  kind  can  be  detonate-, 
providing  simulations  for  training  and  education. 
It  can  even  be  used  to  depict  the  explosion  of  a 
cataract  in  the  simulation  of  a  surgical 
procedure. 

Computer  animation  deals  with  manipulation  of 
three-dimensional  models.  These  models  are 
frequently  represented  as  polygonal  data.  Tha’ 
is,  the  data  is  described  as  a  seiies  of 
coordinate  points  and  a  list  of  polygons  that 
describes  the  relationships  between  t’ ese  points. 
Traditionally  in  computer  animation,  once  the 
object  is  described  there  is  rarely  a  need  to 
break  it  down  into  sub-parts.  During  animation, 
objects  are  transformed  as  a  single  unit  to  create 
the  illusion  of  motion.  For  instance,  a  three 
dimensional  modtl  of  an  rirplane  can  oe  made  to 
fly  through  space  by  incrementally  translating  its 
position  from  frame  to  trnme.  objects  can  be  made 
to  grow  and  shrink  by  the  successive  scaling  of 
the  coordinate  information.  These  transtormat ions 
are  global,  af reeling  all  points  arid  polygons 
uniformly.  The  premise  ot  this  paper  is  that 
since  objects  ale  made  of  component  parts 
(polygons)  there  exists  an  inherent  ability  to 
manipulate  ea  h  polygon  individually  as  if  they 
were  objects  in  themselves. 

The  algorithm  to  control  linear  explosions 
can  be  explained  as  follows.  The  firs'  step  is 
altering  the  description  of  the  data  so  that 
polygons  no  longer  share  poirts  with  adjoining 
polygons,  as  is  normally  the  case.  This  step 
increases  the  number  of  points  in  the  data 
description  without  altering  the  snape  of  the 
object.  Next  a  "movement  vector"  needs  to  be 
calculated  for  each  polygon.  This  vector 
determines  the  direction  of  travel  that  a  polygon 
will  tak6  during  the  course  of  an  animation 
sequence.  It  is  arrived  at  by  subtracting  the 
“center  of  explosion"  from  the  centroid  of  each 
polygon.  The  "center  of  explosion"  is  determined 
by  the  animator  and  is  based  on  the  bounding  box 
of  the  ob  ect. 
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For  i'. stance,  in  the  detonation  of  a  tank  the 
"center  of  explosion"  was  the  minimum  height  of 
the  bounding  box  and  the  center  of  the  object  in 
depth.  This  "center  of  explosion"  caused  the 
pieces  to  fly  in  equal  directions  in  both  the 
hori  ontal  and  u^pth  planes,  but  only  upward  in 
the  vertical  plane,  as  if  the  tank  was  hit  dead 
center  by  some  explosive  force. 

After  the  object  is  redefined  into  unique- 
polygons  and  a  movement  vector  ir  calculated  tor 
each  polygon  the  object  it;  ready  to  be  transform,  d 
by  the  main  program.  The  movement  that  results 
from  this  program  is  based  on  parameters  supplied 
by  the  animator.  These  parameters  determine  how 
much  and  which  transformations  yi 1 1  occur  and 
their  order  of  occurence .  These  pai ameters 
consist  of  a  range  of  minimum  and  maximum  values. 
They  are  computed  internally  by  the  program  using 
a  random  number  generator.  It  is  our  experience 
that  this  randomness  gives  the  movement  a  more 
natural  quality.  It  is  important  that  the  random 
number  generator  be  given  the  same  seed  value  for 
each  new  frame  of  animation  to  insure  the  movement 
continuity  of  each  polygon. 

Within  the  main  program  these  steps  occur. 
First,  each  individual  polygon  is  translated  to 
the  origin.  Then  the  rotations  (if  desired)  are 
performed  in  the  order  specified  by  the  animator. 
The  polygons  are  then  translated  back  to  their 
original  position.  From  this  position  a  new 
location  is  calculated  based  on  the  polygon's 
“movement  vector"  and  parameters  supplied  by  the 
animator.  It  is  at  this  point  that  functions  such 
as  acceleration  and  deceleration  can  be  applied. 
These  translations  can  occur  equally  in  all  three 
coordinate  planes  or  a  unique  value  can  be 
calculated  fur  each  of  the  axes.  For  instance,  if 
the  effect  of  gravity  is  being  simulated  the 
direction  of  movement  in  the  vertical  plane  should 
be  positive  at  first  and  then  gradually  shift  to  a 
negative  translation  until  the  polygon's  position 
is  equal  to  the  level  of  the  ground  plane. 

A  more  precise  mathematical  model  of  an 
explosion  can  be  developed  into  the  present 
program  if  so  desired.  The  first  algorithm 
represented  linear  explosions.  A  second  algorithm 
deals  with  parabolic  explosions.  Objects  launched 
in  a  gravitational  field  at  less  than  escape 
velocity  will  follow  a  parabolic  trajectory.  In 
two  dimer, sions,  and  without  regard  to  friction, 
the  path  of  an  object  along  such  an  arc  if  given 
by 

x  *  t*vO*cos(e)  (1) 

y  -  t*vO*sir.(e)  -  .S*G*t**2,  (2) 

where  e  is  the  angle  of  elevation  above  the  x 
axis,  G  vs  the  acceleration  due  to  gravity  (.98 
meters  per  second),  is  an  initial  velocity,  ard  t 
is  a  parameter.  Since  equations  (1)  and  (2)  give 
the  position  without  regard  to  the  previous 
location  of  the  object,  we  can  extend  (1)  and  (2) 
to  three  dimensions  by  s'raply  specifying  an 
additional  angle  that  gives  a  direction  in 


addition  to  the  elevation.  That  is,  we  tan 
compute  x  and  y  in  the  *  y  -..J  11  ..J  the 

third  coordinate  by  rotating  in  tne  x~  plan,. 

This  gives 

x  -  t*vO*cos( e) *cos( a)  (3) 

z  -  t* vU*  ,os( e)*si r ( a) ,  (A) 

whet t  a  is  the  azimuth  angle,  and  y  is  tound  as  in 


(2)  above.  This  can  be  expressed  conveniently  in 
vector  form: 

x  _  L * vO x 1  (5) 

y  r  t*vU*y’  -  .5  -G*t**2  (6) 

z  =  t*vO*z'  (7) 

where  x’ ,  y' ,  and  z'  are  the  components  of  a 
normalized  vector  giving  an  initial  direction,  vO 
is  an  initial  velocity,  and  t  and  G  are  devined  as 
before. 

The  algorithm  for  parabolic  explosions  is 
analogous  to  that  for  the  linear  case. 

for  each  object  begin 

compute  the  center  of  explosion  CE; 

for  each  polygon  Pi  begin 
compute  the  polygon  centroid.  PC i ; 
find  the  vector  Vi  from  GE  to  PCi; 
compute  vOi ,  the  length  of  Vi; 
normalize  Vi; 
end 

end 

for  each  frame 

for  each  polygon  Pi  begin 

compute  xi,  yi,  and  zi  using  eqns.  (5)-(7) 
compute  the  change  in  xi ,  yi ,  zi  since  the 
las:  frame,  call  this  vector  Di; 
translate  Pi  by  Di; 

end 

While  the  velocity  of  the  exploding  polygons 
is  constant  using  this  algorithm,  it  is  possible 
to  scale  this  velocity  in  several  ways.  We 
initially  assumed  that  the  force  of  the  explosion 
was  constant  in  all  directions.  We  could  however, 
scale  PVi  based  on  say,  the  distance  of  the 
polygon  from  the  centroid,  which  is  just  the 
magnitude  of  Vi,  This  would  have  the  effect  of 
giving  polygons  farther  away  from  the  center  of 
explosion  small  initial  velocity.  Or  we  could 
scale  PVi  based  on  some  measure  of  the  angle 
between  a  polygon,  the  center  of  explosion,  and 
some  reference  axis,  and  thus  give  a  direction  to 
the  blast. 

The  effect  of  the  algorithm  is  to  translate 
each  polygon  along  a  parabolic  trajectory  radially 
outward  from  tne  center  of  the  explosion.  The 
center  of  explosion  can  be  the  centroid  of  the 
object  or  some  othe.  point  chosen  to  achieve  the 
desired  effect.  As  in  the  linear  case,  the 
a/.gorithm  specifies  only  the  position  of  the 
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polygon  and  not  its  orientation,  so  totation 
transformations  must  be  explicitly  applied  it 
desired.  We  can  take  advantage  of  the  fact  that 
motion  in  the  x-z  plane  is  constant,  only  the 
vertical  velocity  of  the  object  varies  with  t. 

Thus  we  can  compute  constant  x  and  z  increments 
once,  and  we  need  only  compute  the  altitude  each 
frame. 

It.  is  a  simple  matter  to  calculate  when  the 
height  of  any  polygon  reaches  0.  If  the  initial 
height  of  the  polygon  is  0,  the  final  value  of  t, 
say  T,  when  the  polygon  again  is  0  is  given  by 

T  =  (2.0  *  vO  *  y’ )  G.  (8) 

If  the  initial  height  of  the  polygon  is  some 
non-zero  value,  say  h,  then  we  need  to  increment  t 
beyond  the  point  at  which  the  final  height  of  the 
polygon  reaches  zero,  since  then  the  object  will 
have  returned  only  to  its  initial  haight  h  above 
the  ground  plane.  This  value  of  T  can  be 
determined  from  the  equation 

.5*G*t**2  -  v0*y ' * t  +  h  =  0,  (9) 

solving  for  t  with  the  quadratic  formula.  Both 
(8)  and  (9)  give  a  unique  value  of  T  for  eacl 
polygon,  as  each  polygon  will  follow  a  different 
path  depending  on  its  ini' ial  direction  vector  and 
velocity.  Since  we  r.n  determine  the  time  and 
position  at  which  the  polygon  should  impact,  we 
can  portray  a  splash,  a  cloud  of  dust,  or  a 
secondary  explosion  at  the  point  of  impact. 
Alternatively,  we  can  u;e  the  direction  and 
velocity  of  the  polygon  during  preceding  frames  to 
compute  a  subsequent  trajectory  and  thus  cause  the 
polygon  to  appear  to  bounce  after  it  struck  the 
ground. 

An  extension  to  these  programs  would  be  the 
ability  to  ~roup  polygons  into  units  and  then 
animate  these  units.  Another  extension  which  we 
implemented  creates  animation  quite  unlike 
explosions  and  implosions.  The  idea  of 
manipulating  polygons  is  the  same.  The  difference 
is  ‘hat  the  data  structure  is  not  altered  into 
unique  polygons.  Instead  the  polygons  are 
triangular! zed,  while  still  sharing  points  with 
neighboring  polygons.  This  step  is  performed  to 
protect  against  aonplanar  polygons  which  typical 
3can  coverting  algorithms  do  not  accept.  With 
this  technique  objects  can  be  made  irregular  and 
new  shapes  can  be  formed.  This  technique  was  used 
to  create  tne  ocean  like  data  seen  in  the 
accompanying  photographs.  Before  modification 
this  "ocean"  was  a  flat  plane.  The  "waves"  were 
created  by  random  rotation  of  constituent 
polygons . 

SUMMARY 

We  have  described  techniques  for  the 
simulation  of  explosions  of  crafts  or  vehicles. 

In  the  paper  we  were  not  concerned  about  the 
precise  physics  involved  in  such  a  problem.  Our 
concern  was  only  with  the  visual  effect.  We  view 
this  effort  as  a  preliminary  effort  and  more  work 
needs  tc  be  done  for  an  Implementation  in  a 


real-time  flight  simulator.  It  is  important  to 
mention  that  much  of  the  realism  associated  with 
our  explosions  is  due  in  part  to  our  display 
algorithm.  Farther  extensions  of  our  techniques 
to  real-time  display  would  reouire  a  careful 
analysis  of  the  relationships,  trade  offs  and 
computational  costs  which  may  be  necessary  among 
various  algorithms  to  achieve  realistic 
explosions. 
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CST,  A  NEW  WAY  TO  REALISTIC  VISUAL  SIMULATION 

by 

Dr.  Robert  Stiokel 
Honeywell  GmbH  Ma i nta 1 /Germany 

ABSTRACTS 

A  new  fully  digital  visual  simulation  principle 
(computer  synthesized  imagery  CSI )  is  described  that 
uses  digitized  image  components  as  terrain,  target, 
sight  reticle  pattern,  and  shell  tracer.  By  means  of  a 
computer  controlled  composition  unit  a  sythesis  of  the 
components  is  carried  out  that  leads  to  a  realistic 
image  of  the  battle  fied.  Outlook  is  given  for  further 
upgrades  of  the  moderate  cost  principle  as  well  as  hints 
for  possible  appl ieations^ 


INTRODUCTION 


Visual  simulation  systems  are  of 
outstanding  dnportance  for  training 
systems  where  the  enhancement  of  the 
trainee  s  capabilities  to  assess,  an!  to 
react  to,  the  inrormation  presented  to 
him  through  his  visual  perception  chcinnel 
plays  a  pivoting  role. 

This,  in  particular,  is  obvious  ^or  any 
kind  of  gunnery  simulators  for  weapon 
systems  with  optical  sights,  flirs,  etc. 
where  target  aquisition  (detection, 
recognition,  identification!  is  a 
significant  phase  of  a  full  engagement. 

At  first  the  most  important  requirement' 
for  a  visual  simulation  system  for  any 
kind  of  gunnery  shall  be  discussed. 

The  first  set  of  requirements  refer  to 
the  presentation  of  the  terrain  image  and 
is  qiven  in  table  1 . 

The  next  set  of  requirements  is 
collected  in  table  ?  and  describes  the 
target  presentation  in  this  terrain 
imaqe . 

Finally  a  set  of  requirements  as  qiven 
in  table  3  depicts  the  influence  of  the 
weapon  system  under  consideration  on  the 
visual  simulation  system. 

Looking  at  these  tables  the  most 
critical  areas  can  be  determined  that 
existing  visual  simulation  principles  are 
suffering  from  either  individually  or  in 
combination.  These  areas  are  given  in 
table  4. 


COMPUTER  SYNTHESIZED  IMAGERY  PRINCIPLES 


In  the  past  two  years  Honeywel 1  has 
tried  to  find  answers  to  the  questions 
raised  in  the  proceed inq  chapter  with  the 
development  of  the  fully  diqital  CSI 
(computer  synthesized  imagery)  principle. 

The  following  paragraphs  shall  give  an 
introduction  into  the  principles  of 
operation  of  the  CSI. 

As  one  can  take  from  the  name  the  basic 
idea  of  CSI  is  to  synthesize  the  image 
which  shall  be  sepn  by  the  trainee  from 
the  typical  imaae  components  as  qiven  in 
table  S . 

The  following  basic  block  diaqram  in 
figure  1  shall  give  an  introduction  in*o 
the  principle  of  the  system. 

This  block  diaqram  shows  four  memories. 

In  the  first  memory,  a  digital  image  of 
the  terrain  is  stored.  A  second  memory 
allows  for  storage  of  all  imaqes  of  on** 
or  more  targets.  A  third  memory  contains 
the  pertinent  sight  reticle  pattern  of 
the  weapon  system  under  consideration, 
and  finally  a  fourth  memory  contains  the 
weapon  d  livery  effects  as  tracer,  burst 
on  target,  etc. 

Each  of  these  memories  delivers  a 
digital  imaqe  information  to  a  digital 
composition  unit  which  is  under  con.rol 
of  a  process  control  computer. 

In  this  unit,  the  afore  mentioned 
synthesis  of  the  images  is  carried  out. 

It  results  in  a  digital  TV  imaqe  with  a 
size  of  GOO  by  GOO  pixels  each  wi  *-h  four 
to  six  bits.  Provided  this  imaqe  is  read 
out,  and  converted  into,  a  video 
according  to  the  television  standards 
(i.e.  with  the  cor respond \ nq  frame  rate), 
a  standard  TV  imaqe  can  be  displayed  on 
the  screen  of  a  television  monitor. 
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TECHNICAL  SOLUTIONS 


After  these  more  general  remarks  some 
detailed  explanations  shall  be  given. 


Terrain  Memory 

First  the  terrain  memory  and  the.  diaital 
terrain  image  will  be  discussed  in  more 
detail  . 

Basis  of  the  digital  terrain  image  is  a 
panorama  foto  taken  by  a  360  degrees 
azimuth  camera  at  any  desired  location  in 
the  real  world.  This  foto  representing  a 
cylinder  projection  of  the  actual  terrain 
is  submitted  to  a  standard  digitization 
process,  the  result  of  which  is  a  digital 
terrain  image  that  can  be  stored  in  a 
computer  mass  memory,  preferably  a 
magnetic  disk.  If  necessary,  image 
editing  can  be  carried  out  to  erase 
scratches  on  the  film,  to  compensate  for 
film  exposure  d i screpanc i es  along  the  363 
degrees  azimuth,  misalignments  of  the 
film  on  the  digitizer,  and  other. 

Because  of  the  limited  reading  speed  of 
the  disk,  the  whole  digital  terrain  image 
is  now  read  into  a  high  speed  random 
access  memory.  From  here,  the  image  is 
now  delivered  to  the  digital  composition 
unit  as  shown  in  figure  1. 

This  process  shall  be  explained  in  more 
det a i 1 s  in  figure  ? . 

This  figure  shows  that  in  a  preparation 
phase  the  random  access  memory  .is  loaded 
with  the  terrain  image 

Nnw,  according  to  the  viewing  direction 
of  the  trainee  and  the  field  of  view  of 
his  optic,  the  corresponding  imaae 
section  is  ad  Iressed  via  the  process 
control  computer,  and  lifted  out  of  the 
random  access  memory.  After  digital  to 
video  conversion,  i-  can  now  be  displayed 
on  the  TV  sc  ret  n . 

It  is  obvious,  that,  to  obtain  standard 
TV  frames,  the  whole  transfer  from  the 
random  access  memory  to  the  TV  screen 
must  be  under  the  control  of  a  TV  clock 
that  runs  the  process  according  to  the  TV 
standards.  This  means,  that  the  random 
access  memory  must  deliver  a  pixel  stream 
with  a  rate  of  a’ccul  60  nanoseconds  per 
pixel.  That  means  also,  that  within  each 
frame  time  a  new  image  is  to  be 
generated,  and  transmitted  to  the  screen. 
By  means  of  the  signals  derived  from  the 


control  handles  of  the  trainee  which 
define  the  section  to  be  displayed  on  the 
screen,  a  continual  motion  cl  the  image 
can  be  achieved. 

Furtheron,  a  special  techique  allows  for 
simultaneous  read  out  of  more  than  one 
image,  even  when  an  overlap  of  the  images 
occurs.  This,  in  turn,  al lows  for  sharing 
the  high  speed  random  access  memory  for  a 
larger  number  of  trainees,  which  gives  an 
optimal  memory  utilization. 

To  obtain  best  resolution,  the  TV  screen 
diameter  is  matched  to  the  field  of  view 
of  the  optic  under  consideration.  This  in 
connection  to  the  azimuth  and  elevation 
gaming  area  determines  the  total  size  of 
the  memory.  Because  of  the  multiple 
access  capability,  however,  only  one 
memory  for  several  users  is  to  be  made 
available. 


Target  Memory 

Next,  the  target  memory  shall  be 
discussed . 

Generally,  the  target  image  can  be 
generated  by  the  well  known  CGI  process. 
This  can  be  either  a  real  time  process, 
that  requires  all  pertinent  special 
purpose  hardware  and  real  time  software, 
or  a  non  real  time  process.  In  the  latter 
case,  all  target  perspective  views  that 
make  up  a  complete  tarqat  trajectory  of  a 
moving  target,  are  generated  and  stored 
in  the  target  memory  as  shown  in  figure  3 
toqether  with  a  set  of  addresses  that 
determines  at  which  area  of  the  terrain 
the  target  shall  appear. 

These  views,  now  interpreted  as 
consecutive  frames,  are  read  from  the 
memory  into  the  composition  unit  with  a 
standard  TV  frame  rate.  That  means,  that 
alike  the  terrain  memory,  the  target 
memory  delivers  a  pixel  stream  the  timing 
of  which  is  identical  to  that  of  the  high 
speed  random  access  memory  for  the 
terrain . 

Regarding  the  size  of  the  memory,  the 
following  corsi derat i on  can  be  made. 

Only  in  some  more  unrealistic 
constellations,  the  tarqet  would  have  the 
full  size  of  the  field  of  view.  However, 
taking  realistic  engagement  ranges  and 
target,  sizes  into  consideration,  it  can 
easily  be  determined,  that  the  target 
size  in  relation  to  the  field  of  view 
rarely  exceeds  10  ?  of  the  field  of  view. 
Thus,  with  the  field  of  view  being 
represented  by  600  vertical  TV  lines,  a 
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target  would  be  represented  by  about  60 
lines  at  maximum. 

This,  in  turn,  leads  to  a  practical 
target  memory  size  of  64  by  64  pixels  per 
target  frame.  While  for  the  terrain  a 
pixel  size  of  6  bits  turns  out  to  be  most 
adequate,  a  target  pixel  size  of  4  bits 
is  fully  satisfactory. 

Sight  Reticle  Pattern  Memory 

It  appears  more  practical  to  generate  a 
reticle  pattern  on  the  TV  screen  rather 
than  to  use  an  optical  reticle  in  front 
of  the  screen.  Thus  any  mechanical 
adjustment  can  be  avoided. 

In  comparison  to  the  terrain  and  target 
memory,  the  sight  reticle  memory  is 
fairly  simple.  Its  size  is  600  by  600 
pixels  with  only  one  bit  per  pixel.  Such 
a  memory  can  be  loaded  with  any  kind  of 
sight  reticle  pattern. 

Because  some  sights  have  variable  reticle 
parts,  as  for  instance  the  turret 
position  indicator  of  a  tank  sight,  two 
memory  planes  are  used  which  can  be 
shifted  with  respect  to  one  each  other  as 
shown  in  figure  4. 


Tracer  Memory 

The  tracer  memory  also  is  a  simple 
memory.  Tt  contains  only  one  pixel  per 
round  with  4  bit  for  each  pixel  to  allow 
for  brightness  variation  in  accordance 
with  the  increasing  distance  to  the 
observer . 

The  addresses  of  the  tracer  pixel  are 
calculated  in  the  process  control 
computer  according  to  the  shell 
trajectory  and  the  parallax  angle. 


Burst  on  Target,  Memory 

The  burst  on  target  memory  has  a  size  of 
e.g.  3  by  3  pixels  with  each  one  bit.  'f'he 
address  of  tae  center  pixel  is  identical 
to  that  of  the  tracer  pixel.  The  burst  or 
target,  memory,  however,  is  only  activated 
if  a  tiit  occurs,  and  than  only  for  one  or 
two  frames.  Thus,  a  short  flash  on  the 
target  ('an  be  made  visible. 


Image  Compos v r i on  bn  it 

The  proceeding  paragraphs  described, 

"hat  each  imaqe  component  is  delivered  as 
a  pixel  stream  from  its  memory.  The 
composition  process,  now,  is  controlled 


by  two  main  parameters  which  are 
addresses  and  priorities. 

This  shall  be  demonstrated  by  means  of 
the  most  complex  composition  process  for 
terrain  and  targets. 

First,  the  position  of  the  target  in  the 
terrain  can  be  easily  determined  by  a  set 
of  addresses  of  consecutive  target  frames 
in  reference  to  the  terrain  addresses. 
This  process  is  depicted  in  figure  5. 
Whenever  the  terrain  addresses  reach  the 
addresses  of  the  target,  the  composition 
unit  is  armed  to  switch  to  the  pixel 
stream  of  the  target  memory.  'T’he 
switching  finally  is  carried  out  when  a 
target  pixel  occurs  that  deviates  from 
zero. 

Second,  the  priority  control  shall  be 
invest igated . 

To  achieve  high  realism,  a  target  must 
either  appear  before  a  terrain  feature  as 
for  instance  a  bush,  when  it  is  closer  to 
this,  or  behind  a  terrain  feature,  when 
ic  is  further  away.  In  terms  of  priority, 
this  means  that  either  the  target  pixels 
have  higher  priority  than  the  terrain 
pixels  (thus  the  target  obscures  the 
terrain  behind),  or  the  terrain  pixels 
have  higher  priority  which  results  in  a 
partial  or  a  full  masking  of  the  target 
by  the  terrain  feature. 

Now,  with  a  given  target  trajectory  in 
the  terrain,  it  is  fairly  easy  for  an 
observer  to  visually  determine  those 
features  which  may  obscure  the  target. 
This  features  arc-  marked  on  the  screen  in 
a  preparatory  process  as  so  called 
obscuration  lines  as  shown  in  figure  6. 
This  process,  in  turn,  generates  a  set  o* 
priority  bits  that  is  associated  to  the 
addresses  of  the  terrain  pixels,  and  t ha; 
finally  controls  whether  a  terrain  or  a 
target  pixel  is  to  be  displayed. 

As  the  degree  of  detail  of  ♦'he  the 
obscuration  lines  goes  down  to  the  pixel 
level  ,  the  addresses  of  t-ho  target 
obscuration  can  be  obtained  to  the  same 
deqroe.  This,  for  instance,  allows  oven 
for  a  target  being  partially  visible 
through  the  branches  of  trees  . 

Similar  processes,  as  de.se  ribed  in  the 
proceeding  paragraphs,  finally  are  used 
to  determine  the  visibility  of  the 
reticle  sight  pattern  and  the  tracer. 


Special  Effects 

look  up  table  terhnioues  are  used  to 
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allow  for  following  effects: 

daylight  conditions 
visibility  (fog/haze) 
gun  smoke 

Likewise,  in  ronnection  with  some  more 
complex  transfer  functions,  look  up 
tables  even  allow  for  the  simulation  of 
the  thermal  image. 


OUTLOOK 


Tn  the  preceeding  paragraphs,  the 
presen’  status  of  the  CS1  visual 
simulation  system  capabilities  is 
described.  Beyond  that,  the  following 
enhancements  can  be  integrated  into  the 
excisting  design. 

Color 

Being  based  on  the  TV  principle  the 
black  and  white  CSX  system  can  be 
upgraded  •  ^  a  color  system  by  means  of 
adequate  memory  space.  This  in  principle 
does  not  affect  the  composition  process. 


Battle  Field  Smoke 

Tn  a  similar  way  as  targets,  smoke 
columns  can  be  planted  into  the  terrain 
with  variable  ascent  angle  being 
determined  by  a  wind  velocity  datum. 


Terrain  Impact 

Dust  clouds  at  a  terrain  impact  of  a 
shell  can  be  planted  into  the  image  again 
by  means  of  similar  techniques,  while 
time  controlled  look  up  tables  provide 
the  trpical  fade  away. 

Target  Dust  clouds 

Dust  clouds  appearing  behind  mc-’inq 
targets  finally  can  be  generated  also  in 
a  similar  way  as  those  at  the  impact 
jroint  of  the  shel  1  . 


User  Designed  Target  Trajectories  and 

Ss.er.  arias _ 

It  appears  highly  desirable  tha*  the 
user  of  a  training  system,  i.e.  the 
instructor,  has  means  to  create  target 
trajectories  and  scenarios  in  accordance 
with  the  given  training  doctrines. 
Provided  a  sufficient  target  data  base 


is  stored  on  the  system  disk  this 
important  feature  can  be  made  available 
by  means  of  an  extra  software  package. 
This  package  will  allow  the  instructor  to 
plant  a  target  at  any  desired  spot  into 
the  terrain  and  match  its  size  and 
orientation  to  the  given  terrain 
conditions  at  the  selected  spot.  By  means 
of  interpolation  routines,  a  sequence  of 
such  targets  can  be  concerted  by  che 
system  computer  into  consecutive  target 
francs  thus  resulting  in  a  new  moving 
target  trajectory. 


CONCLUSION 


The  CSI  visual  simulation  principle  as 
described  in  the  preceeding  paragraphs 
seems  to  be  a  useful  contribution  to  the 
family  of  existing  visual  simulation 
systems.  With  its  fully  digital  process 
it  avoids  any  mechanical  component  and 
allows  for  the  application  in  rougn 
environment . 

For  most  of  the  processes  being  software 
controlled  the  reproduction  of  a  CSI 
system  ranqes  in  a  moderate  price  level . 

Finally  with  its  realistic  images,  CSI 
seems  to  be  an  ideal  visual  simulation 
for  any  kind  of  weapon  systems  ranging 
from  tanks  over  anti  aircraft  guns, 
missile  launchers  to  hand  held  weapons  as 
bazookas,  wire  guided  anti,  tank  missiles 
or  comparable. 
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TABLE  1 


TABLE  4 


REQURI REMENTS  FOR  TERRAIN  IMAGE 
TERRAIN  IMAGE 

0  MUST  BE  REALISTIC 

0  MUST  BE  ARBITRARY 

0  MUST  BE  EASILY  EXCHANGEABLE 

0  MUST  HAVE  WIDE  GAMING  AREA 

0  MUST  HAVE  VARIABLE  LIGHTING  CONDITIONS 

0  MUST  HAVE  COLOR 

0  MUST  HAVE  BATTLEFIELD  SMOKE 

I  ABLE  2 

REQUIREMENTS  FOR  TARGET  IMAGE 

0  TARGETS  MUST  BE  REALISTIC 
0  TARGETS  MUST  MOVE  R _ALI ST  I CALLY 
0  TARGETS  MUST  HAVE  REALISTIC  MASKING 
0  TARGETS  OF  ANY  KIND  MUST  BE  AVAILABLE 
0  TARGETS  MUST  APPEAR  IN  GROUPS 
0  TARGET  SCENARIOS  MUST  BE  USER  DESIGNED 
0  TARGETS  MUST  GENERATE  DUSTCLOUDS 
0  TARGETS  MUST  GENERATE  MUZZLE  FLASHES 

TABLE  3 

GENERAL  VISUAL  SIMULATION  REQUIREMENTS 
VISUAL  SIMULATION  MUST  ALLOW  FOR 

O  SIMULATION  OF  FlIR 
0  VARIABLE  SIGHT  MAGNIFICATIONS 
0  IMAGE  AS  SEEN  WITH  NAKED  EYE 
0  SIMULATION  OF  WEAPON 

-  GUN  SMOKE 

-  SHELL  TRACER 

-  TERRAIN  IMPACT 

-  BURST  ON  TARGET 

-  RICOCHET 


DEFICIENCIES  OF 

EXISTING  VISUAL  SIMULATION  PRINCIPLES 

0  UNREALISTIC  TERRAIN  IMAGE 
0  EXPENSIVE  TERRAIN  DATA  BASE 
0  LIMITED  GAMING  AREA 
0  UNREALISTIC  TARGETS 
0  UNREALISTIC  TARGET  MASKING 
0  NO  USER  DESIGNED  TARGET  SCENARIOS 


TABLE  5 

COMPONENTS  FOR  CSI  VISUAL  SIMULATION 

0  TERRAIN  IMAGE  :  TABLE  1 

0  TARGET  IMAGE  :  TABLE  2 

0  SIGHT  RETICLE  PATTERN  :  TABLE  3 
0  WEAPON  DELIVERY  EFFECTS:  TABLE  A 
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ABSTRACT 

This  paper  reviews  recently  developed  training  devices  and  techniques  used  by  the  Warsaw 
Pact  countries  (WL’C)  to  improve  the  combat  readiness  and  effectiveness  of  their  ground  forces. 
Most  of  these  devices  are  relatively  simple,  straightforward,  and  tailored  to  the  tactical 
doctrine  of  the  Warsaw  Pact  forces;  however,  Soviet  literature  has  recently  mentioned  the  use 
of  laser  simulation  equipment  as  a  training  device  for  gunnery  training.  The  use  of  mockups 
and  of  special  simulation  equipment  and  techniques  is  emphasized,  particularly  in  tank- 
gunnery,  chemical-biological-radiological,  and  a ir-mobi le-t roop  training. 


INTRODUCTION 


The  high  performance  and  destructive  power 
of  modern  weapons  and  equipment  necessitate  that 
operators  be  able  to  respond  rapidly,  aim  with 
extreme  accuracy,  and  have  a  thorough  knowledge 
of  the  effectiveness  and  operational  limits  of 
combat  systems.  To  achieve  these  results, 
military  leaders  of  the  WPG  believe  that  the  best 
training  consists  of  repetitive  functions  under 
varying  conditions,  with  a  gradual  increase  in 
the  degree  of  difficulty.  This  procedure  best 
familiarizes  the  trainees  with  the  weapons  or 
equipment,  ensuring  maximum  of fectiveness  in 
their  use.  As  t’.ieir  unit  commanders  seek  more 
efficient  methods  of  preparing  troops  for  combat 
operations,  simulation  techniques  and  devices 
receive  more  attention.  The  increase  in  the 
variety  of  teaching  aids,  mockups  of  actual 
equipment,  miniature  training  grounds,  and 
innovative  methods  and  techniques  is  certainly 
intended  to  provide  for  a  higher  quality  of 
training.  In  classrooms  and  demonstration 
exercises,  instructors  use  mockup  posters, 
diagrams,  films,  and  slides.  Simulators  and 
mockups  are  used  when  the  trainees  are  considered 
proficient  in  the  basic  techniques.  Many 
trai  ring  areas  contain  small  terrain  mockups 
equipped  with  miniature  terrain  features,  manmade 
structures,  and  equipment  for  use  by  the  trainees 
to  further  their  training  in  practical  problems. 

The  training  programs  of  the  non-Soviet  WPC 
ground  forces  are  essentially  identical  with  that 
of  the  Soviets.  They  differ  primarily  iu 
standards  of  qualification  and  in  the  types  of 
training  aids  and  simulators.  The  types  of 
training  devices  developed  for  use  by  tne  various 
branches  of  service  vary  from  complex  sophisti¬ 
cated  simulation  equipment  to  locally  fabricated 
simple  training  aids. 

DISCUSSION 

fan k  Training  SlmulatLo n 

Extensive  training  is  conducted  on 
simulators  during  tank  gunnery  training  before 
trainees  actually  operate  the  tanks  or  fire  the 
main  and  secondary  armament  on  standard  ranges. 
Unit  commanders  arc  allowed  a  wide  latitude  in 
selecting  and  developing  training  aids.  Among 


the  devices  used  in  training  tank  crews  are  gun 
trainers  to  improve  sighting  and  aiming  skills; 
trainers  with  sights,  instrument  panels,  and 
instructor's  control  panels;  and  tank  trainers  on 
motion  simulator  platforms  to  simulate  cross¬ 
country  movement.  Newer  Soviet  tanks  have 
elaborate  training  simulators.  For  the  gunner  a 
cutaway  turret  (fig  1)  includes  the  fire  control 
and  automatic  loadc.r.  A  trainer  consisting  of  a 
hull  on  which  a  turret  is  mounted  (fig  2)  is  used 
to  train  the  tank  crews  in  methods  of  conducting 
cire  against  fixed,  Jisappearing,  and  moving 
targets  during  the  day  and  at  night.  The  turret 
includes  a  mockup  of  a  tank  gun,  a  stabilizer,  a 
training  machinegun,  a  gunner's  night  sight,  a 
commander's  periscope,  and  a  tank  intercom  system. 
The  tank  rocking  frame  device  consists  of  a  metal 
frame  onto  which  a  tank  may  be  placed  and  rocked 
to  simulate  cross-country  movements.  It  is  used 
in  conjunction  with  sighting  and  gun-stabilization 
devices.  The  frames  are  usually  Installed  in 
classrooms,  cantonment  areas,  and  subcaliber 
ranges . 


Figure  1.  Training  Turret 
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i  g  u  r  l*  2  .  *‘-72  Tan k  T minor  W i  t  h 

An: omaf ic  Loader 

N  amorous  training  aids  have  a !  so  been 
cL-ve  i  opod  tor  r  a  nk  drivers.  Those  trainers  range 
frr:!  a  simple  device  tor  teaching  the  rules  of 
to.e  road  to  an  elaborate  driving  s  i m* ‘later  that 
provides  the  trainee  with  a  dupl  iea  t  r  the 
actual,  driver’s  compartment ,  which  pitches  and 
rolls  to  match  file  terrain  projected  on  a  screen 
in  ! ront  of  the  trainer.  These  elaborate 
simulators  also  create  engine  noises  that  result 
from,  the  trainee’s  use  of  the  gear  selector  and 
his  man i pul  at  Lon  ef  the  accelerator. 

An  Artillery  Weapon  Simulator 

The  military  forces  of  the  LTC  consider  the 
multiple  rocket  launcher  (MKL)  a  principal 
artillery  weapon,  and  each  crew  member  must  be 
trained  to  know  its  capabilities  and  performance 
characteristics.  During  specialized  training,  the 
classrooms  in  which  the  training  is  performed  are 
divided  into  two  sections,  one  for  theoretical 
study  and  the  other  for  practical  exercises. 

Theory  is  taught  in  a  classroom  equipped  with 
training  aids,  including  the  latest,  models  of 
audio  and  visual  devices,  recorders,  and  pro¬ 
jection  equipment.  Practical  training  is  con¬ 
ducted  on  a  mocktip  weapon  installed  on  a  special 
t  mine  that  provides  the  students  with  full-circle 
accessibility.  All  of  the  control  and  functional 
elements  are  exactly  like  those  of  the  actual 
weapon.  A  training  mockup  of  the  Czechoslovak 
M-70  MR1.  is  shown  in  figure  3.  The  students  are 
taught  start-up,  operating,  and  shut-down 
procedures  that  they  will  perform  on  the  weapon. 
All  electrical-  and  hydraulic-powered  elevating, 
traversing,  and  reloading  components  and  controls 
are  accessible  for  maintenance,  repair,  and 
adjustment  through  removable  plexiglass  covers. 
During  the  instruction  periods,  malfunctions  in 
the  weapon  are  purposely  introduced,  and  students 
are  required  to  locate  and  eliminate  the 
mechanical  source  of  the  problem  until  the  system 
is  operating  properly.  The  MRL  simulator  is 
considered  to  be  an  important  development  for 
providing  realistic  progressive  training  for  the 
crew  in  the  operation  of  the  weapon  system. 


Figure  3.  Czechoslovak  MRL 
Training  Simulator 

If ijjh •  Training  With  Simulators 

The  WPG  small-arms  training  methodology  has 
remained  basically  unchanged  since  World  War  11. 
Their  military  forces  have  been  able  to  produce 
effective  infantrymen  ready  for  combat  because 
ot  their  effective  rifle  training,  which  closely 
resembles  combat  conditions.  Training  scenarios 
simulate  the  manner  in  which  the  ground  forces 
plan  to  employ  small  arms  in  combat.  Al* hough 
basic  methodology  has  not  changed,  the  WPC 
emphasize  new  training  techniques,  hardware,  and 
devices  in  an  effort  to  produce  superior  riflemen 
who  are  ready  for  combat.  They  have  developed 
devices  such  as  liming  devices,  and  optical 
devices  to  be  used  in  rifle  training  programs. 

BMP  .armored  vehicle  gunnery  training  is 
emphasized,  and  a  trainer  for  BMP  personnel  is 
available.  Several  electro-optical  devices  that 
use  laser  or  light  beams  to  simulate  bullets  have 
been  developed  for  use  in  this  training.  In  most 
of  the  items,  photocells  are  installed  on  the 
target  to  register  a  "hit.”  The  primary 
advantages  of  these  devices  are  that  they  are 
completely  safe,  relatively  inexpensive  to  pro¬ 
duce,  and  take  up  little  space.  Some  are  designed 
for  classroom  use,  others  for  ranges. 

The  electric  aiming  device,  (fig  4)  was 
invented  by  a  soldier  in  a  Soviet  military  unit 
and  is  used  by  that  unit;  such  innovations  are 
encouraged.  The  device  uses  electro-mechanical 
equipment  in  conjunction  with  on  aiming  stand, 
which  is  free  to  rotate  both  vertically  and 
horizontally.  The  trainee's  rifle  is  fastened  to 
the  base,  a  solenoid  sensor  is  attached  to  the 
muzzle,  and  a  wire  is  attached  to  the  trigger. 

An  instructor  performs  the  initial  .alignment; 
then,  the  rifle  is  turned  off  target.  When  the 
trainee  aims  and  then  "fires,"  the  weapon  is 
fixed  in  place.  If  the  shot  is  accurate,  the 
checkir.g  light  on  the  range  officer's  panel  is 
not  illuminated.  In  case  of  n  miss,  however,  the 
lamps  light;  the  deviation  1»  us  eel  to  Judge  the 
aiming*  error. 
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Figure  4.  H  eetronic  Aiming  Device 

Pregrams  Co  teach  infant  rysnen  the  necessary 
*k  ills  *:o  ongagi  aerial  tarn, eta  are  incorpor  ,*tcd 
into  the  rifle  training  program.  Soviet  writings 
'  "ui  ic  a.  t  e  that  there  are  distinct  stages  of 
progress  ion  in  teaching  troops  to  operate  against 
aerial  targets,  and  mock ups  have  been  estahl  ished 
at  va-iou--  ranges  to  simulate  these  actions. 
Advanced  training  tor  air-mobile  troops  involves 
tiring  at  ground  targets  from  inside  he  Li  cop tors. 
Mi>  c  kit s  provided  at  air-mobile  t  ra  in  ing  “ranges 
are  used  to  familiarize  trainees  with  helicopters., 
and  to  provide  practical  experience  in  firing  on 
aerial  targets. 

An  Antiaircraft  Missile  Simulator 


Relatively  simple  in  design  and  operation, 
flirs  an  t  i  a  i  rc  ryrr  c  missile  simulator  (fig  5)  wras 
developed  terrain  operators  in  the  firing  of  a 
portable^' ant  Lai  re  raft  missile  equipped  with  an 
infrared  passive  guidance  system  and  intended  for 
use  again.'t  low-flving  enemy  aircraft.  The 
simulation  equipment  is  mounted  on  the  actual 
weapon,  allowing  Lin.  operator  to  make  use  of  the 
normal  weapon  system  controls  while  carrying  out 
simulated,  practice  firing  continuously.  During  a 
simulated  firing  operation  an  aircraft  target 
generated  by  the  simulator  imago  projector 
enables  the  operator  looking  through  this  aiming 
sight  to  see  the  target  projected  on  a  screen. 
Visual  and  audio  signals  transmitted  at  the 
instant  of  firing  indicate  that  the  target  has 
been  ’ locked  on”  and  the  operator  needs  only  to 
re  lease  the  trigger  mechanism  to  fire  a  simulated 
miss  ill.1  lor  a  hit  or  miss.  During  the  course  of 
i  nv:  t  rue  t  i  on  the  proper  procedures  for  firing  are 
supervised  oy  an  instructor  standing  by. 
instrument  readouts  at  the  moment  of  launch  pro¬ 
vide  the  instructor  with  results  for  scoring  and 
evaluating  the  progress  of  the  trainee-operator. 


Figure  5.  Simulator  for  Soviet 
Antiaircraft  Missile  SysLem 


Comb  at  Engineer  ir a  i  tiers 


Combat  engineer  training  emphasizes  the  use 
of  s i mu  la  t  ‘ on  equipment.  Two  recently  described 
operator  miners  simulating  the  BAT/M  dozer  and 
the  MDK-2  ditching  machine?  (fig  6  and  7/  have 
been  developed  by  the  Soviets.  The  dozer  is  used 
primarily  as  a  route  clearer  and  consists  of  a 
hydraulically  operated  dozer  blade  mounted  on  the 
i rent  of  the  AT-T  heavy  tracked  artillery  tractor. 
The  dozer  trainee  has  a  trainee's  seat  equipped 
with  simulated  crane  hand  controls,  working 
elements,  and  a  dozer  blade.  Included  in  the 
equipment  are  an  error-display  panel  and  an 
instructor's  panel  with  Indicators  and  controls 
for  monitoring  the  exercise.  The  MDK-2M  o itching 
machine  can  entrench  command  posts,  troops,  guns, 
tanks,  vehicles,  and  equipment.  Its  trainer  has 
a  simulated  dozer  Made  and  tremhing  components, 
a  trainee';;  seat,  and  an  error-display  pane].  The 
engineer  equipment  trainers  are  prime  examples 
of  the  Soviet  use  of  Mmularion  technology. 

Much  of  the  extensive  training  required  of  the 
operators  can  be  conducted  in  a  controlled 
learning  environment  with  simulated  practice. 

While  the  use  of  there  trainers  does  not  eliminate 
the  use  of  actual  equipment,  the  Soviets  claim 
that  the  training  period  on  the  engineer  equipment 
itself  is  reduced  by  as  much  as  25%.  This 
reduction  although  fairly  modest,  would  help  to 
'extend  the  service  l.lf''  of  the  equipment,  decreare 
training  time,  and  provide  cost  savings  in  fuel 
and  maintenanefc  so  that,  in  turn,  the  resulting 
savings  in  military  training  costs  would  be 
sign  if  leant . 
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it1  i  r  effort  to  uropare  v domical  units  to 
ffuctively  in  an  environment  contaminated 
.’ills,  the  WPC  m  i  L  Ltarv  leaders  have 
1  the  use  of  training  dev  ices  and  special 
i.  techniques.  The  ability  of  their 
operate  effectively  in  a  CiiR-contnmi- 
i  rontc'.o'.L  is.  of  giont  concern,  and  troop 
:y  in  reconnaissance,  detection,  and 
'.mien  one -at  ions  is  strong Ly  emphasized. 
i  i  ■>  t.  ra  i  n  i  ng  i  s  >*  i  ven  i  -t  t  lie  c  1  ass  room 
ip  lemented  with  training;  aids  and 
Films,  slides,  and  electronic  aids  are 
j1.  in  demonstrating  techniques.  CBR 
seems  to  he  (.concerned  primarily  with 
and  nuclear  warfare.  Several  devices 
simulant  agents  (to  contaminate  terrain 
ie  ;  for  use  in  training)  have  been 
and  discussed  in  Soviet  publications. 
t:e  chemical  contamination  for  training 
the  Soviets  occasionally  use  a  detector 
device  that  contains  tubes  for 
!ig  various  CW  agents.  Hydrogen  cyanide 
rd  tubes  have  been  mentioned,  and  ILquid 
s  frequently  used  as  a  CW  agent  simulant, 
devices  are  frequently  used  to  simulate 
explosion.  A  device  described  art  a 
cloud  simulator  is  usually  associated 
id  to  simulate  radiation  fallout 

Tiie  device  simulates  the  ascent  of  the 
m  near  ground  level,  after  n  nuclear 
.  With  the  aid  of  the  simulator  the 
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Figure  8.  TMS-65  Decontamination 
,  Vehicle  Trainer 

Electronic  Systems  and  Simulation 

To  enhance  the  effectiveness  of  training  in 
an  electronic  warfare  environment,  WPC  commando  "-s 
give  considerable  attention  to  training  with  the 
aid  ol  simulation  equipment.  Radar  simulators 
have  been  available  for  almost  as  long  as  the 
radars  themselves.  Reportedly  unaer  development 
is  a  system  that  uses  digital  computers  to  improve 
the  radar  operator’s  training  skills.  A  radar 
classroom  which  is  an  exact  replica  of  the  Inside 
of  a  radar  station  has  been  reported.  It  Is  said 
to  have  simulation  equipment  that  allows 
.situations  closely  resembling  real  combat  to  be 
displayed  on  the  indicator  screens.  The  impor¬ 
tance  of  acquiring  skills  in  handling  simulators 
and  trainers  to  mpet  the  requirements  of  effec¬ 
tive  combat  training  has  been  emphasized. 

CONCLUSIONS 

Training  devices  are  playing  an  increasingly 
important  rule  in  the  efforts  of  the  WPC  ground 
forces  to  successfully  achieve  their  training 
objectives  of  combat  readiness.  Their  efforts  to 
improve  and  provide  higher-quality  training 
through  the  use  of  simulation  technology  and 
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craining  devices  are  well  known.  The  role  of  the 
training  device  as  an  essential  tool  for 
repetitive  training  under  realistic  combat  con¬ 
ditions  is  becoming  increasingly  important  to  the 
WPC  military  leaders.  As  new  and  more  sophisti¬ 
cated  technology  is  introduced  into  combat  equip¬ 
ment,  training  requirements  become  more  intense 
and  complex.  These  events  lead  to  the  intro¬ 
duction  of  new  tv>es  of  simulators  and  training 
devices 
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ABSTRACT 


^A  new,  effective,  and  low-cost  battal ion(BN)-level  battle  simulation  system,  the  MACE/ 
Micro-Disc  System,  has  been  enveloped,  under  the  sponsorship  of  the  US  Army  Combined  Arms 
Training  Development  Activity  (CATRADA)  at  ft.  Leavenworth,  Kansas.  >The  MACE  simulation  is 
an  improved  version  of  the  Army's  Computer  Assisted  Map  Maneuver  Simulation  I  (CAMMS-I), 
modified  to  operate  on  a  low  cost  personal  microcomputer-videodisc  (Micro-Disc)  system.  Map 
boards  are  replaced  with  military  maps  stored  o.i  a  videocisc  and  displayed  on  color  moni¬ 
tors. _ UjtU  location,  status  and  action  are  shown  in  a  graphical  map  overlay  generated  by 

the  microcomputers .  Operators/ana^ys* s  can  examine  the  simulated  battle  situation  by 
scrolling  the  map  using  a  joystick.  The  overlay  is  keyed  to  map  coordinates  and  scrolls 
with  it.  In  addition,  the  operator  can  zoom  to  focus  on  a  selected  area.  The  areas  dis¬ 
played  can  range  from  4  to  40  km  on  a  side.  The  Micro-Disc  System  employs  distributed 
processing  supported  by  multiple  personal  microcomputers  and  video-disc  players  sharing  a 
common  hard  disk  mass  storage  unit.  This  distributed  processing  architecture  and  local  area 
networking  permits  graceful  degradation  in  the  event  of  failure  of  one  of  the  microcompu¬ 
ters;  an  exercise  can  continue  to  function  with  other  microcomputers  in  the  system  perform¬ 
ing  the  functions  of  the  disabled  microcomputer.  Local  area  networking  also  permits  easy 
expansion  of  the  system  for  more  complex  simulations.  Operational  tests  of  the  system  by 
the  9th  Infantry  Division  at  Ft.  Lewis,  Washington  have  produced  results  exceeding 
expectations. 


BATTALION-LEVEL  COMMAND 
AND  CONTROL  TRAINING 

Current  Approaches 

Rapidly  evolving  microcomputer,  distributed 
processing,  and  videodisc  technologies  have  been 
integrated  into  an  effective,  efficient  and  low- 
cost  battalion  (BN)-level  command  and  control 
(C^)  training  system--the  MACE/Micro-Di sc  System. 
This  system  can  meet  critical  Army  needs  in 
BN-level  C?  training. 

BN-levei  training  in  the  US  Army  currently 
is  limited  to  one  or  two  CPXs  per  year,  typically 
manual.  Selected  CONUS  units  travel  to 
Ft.  Leavenworth,  Kansas,  once  every  three  or  four 
years  to  use  the  Combined  Arms  Tactical  Training 
System  (CATTS).  There  is  some  use  of  board 
games,  and  some  use  of  the  Computer  Assisted  Map 
Maneuver  Simulation  I  ( CAMMS- I ) .  While  each  of 
these  approaches  can  provide  valuable  training, 
their  limited  use  can  be  attributed  to  specific 
shortcomings : 

(1)  Manual  CPXs  are  people-intensive, 
requiring  j  sizeable  staff  of  control¬ 
lers  to  produce  a  realistic  exercise. 
Battle  outcomes  are  decided  by  the  sub¬ 
jective  judgment  of  the  controllers. 

(2)  Board  games  are  also  people-intensive, 
and  are  vulnerable  to  dominant  person¬ 
alities  among  the  players.  There  is 
again  no  objective  source  of  resolution 
of  scenario  conflicts. 


(2)  CAMMS  is  objective,  effective,  and 
available,  but  still  uses  map  boards 
and  does  not  work  in  real  time.  In 
addition  it  must  be  operuted  via  a 
dial-up  telephone  circuit  to  Minneapo¬ 
lis,  and  other  locations.  This  is 
expensive  and  difficult  to  schedule. 

(4)  Except  at  CATTS,  a  realistic,  stress- 
filled  environment  is  difficult  to 
maint  in.  The  fidelity  of  the  TOC 
setting,  and  of  the  interaction  of  the 
BN  comander  and  his  staff  with  the  BN 
elements  playing  the  scenario  are 
essential  to  effective  training. 

(5)  CATTS  training  is  effective  and  realis- 
tir,  but  expensive  because  of  the 
travel,  and  has  insufricient  availabil¬ 
ity  to  meet  training  needs. 

With  the  training  system  located  on-site,  BN 
staff  training  programs  can  be  expanded.  It  will 
be  easier  to  schedule  training  sessions,  and 
easier  to  reschedule  if  BN  plans  have  to  change 
for  any  reason.  In  addition,  new  training  capa¬ 
bilities  can  be  developed  which  use  the  MACE/ 
Micro-Disc  System  and  are  tailored  to  local 
needs.  These  advantages  are  gained  at  low  cost. 
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me  i<-  training  Kequirement 

BN  readiness  requires  that  C?  training  exer¬ 
cise  the  functions  performed  by  the  BN  commander 
and  his  staff  in  a  realistic  stressing  environ¬ 
ment.  Emphas;:.  must  be  placed  on  command  deci¬ 
sions,  control  from  brigade  (BDE)  through  BN 
staff  to  companies  and  other  BN  support  elements, 
coordination  among  BN  elements,  information  man¬ 
agement  and  use,  and  the  use  of  BN  assets. 

Exercising  C?  functions  does  not  require 
fully  detailed  tactical  resolution  (e.g..  line  of 
sight  determination  for  individual  weapon  sys¬ 
tems).  It  does  require  accurate  replication  of 
t Me  flow  of  information  on  the  battle  situation 
and  the  interaction  between  the  BN  commander,  his 
staff,  BPE  headquarters,  and  BN  elements.  The 
simulated  sequence  of  events  in  th»  battle  exter¬ 
nal  to  BN  headquarters  must  be  consistent  with  a 
realistic  scenario. 


pUit  vcrilUCi.  r\  jump  i  ,  i  tu  •  v*  ^  - - -  -  , 

also  provided  for  use  as  needed  to  be  consistent 
with  the  situation  in  the  battle  simulation  (Fig¬ 
ure  2 ) . 


Simulation  Center. 


C?  in  battle  is  highly  stressing.  There¬ 
fore,  a  training  simulator  must  stimulate  the 
players  both  physically  and  mentally.  On  the 
physical  level,  the  play  area  must  look  and  sound 
like  a  tactical  command  post  near  the  front. 
From  the*  mental  perspective,  the  effects  of 
orders,  and  the  results  of  conflicts  between 
friendly  forces  and  the  0°F0R  must  be  both  objec¬ 
tively  determined  and  perceived  as  objective. 
This  requirement  calls  for  a  computer-driven  sim¬ 
ulation  that  can  quickly  determine  the  results  of 
complex  battle  situations  based  on  the  relative 
strengths  of  units  and  their  weapons. 

Finally,  local  operation  of  the  simulation 
is  needed  to  assure  <.asy  and  frequent  access  by 
individual  BNs.  Each  BN  should  tr’in  several 
times  a  year.  Local  control  makes  this  possible 
and  simplifies  rescheduling  when  it  is  needed. 

THE  BATTLE  SIMULATION 
CENTER  CONCEPT 

C?  train  ing  requirements  can  be  met  by  the 
Battle  Simulation  Center  (BSC).  The  essential 
feature  of  the  BSC  (Figure  1)  is  that  it  places 
the  BN  commander  and  his  staff  in  a  situation 
that  closely  simulates  actual  stress-fil led  bat¬ 
tle  conditions.  The  commander  and  his  staff 

operate  out  of  M577  mockups  or  actual  command 
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figure  1.  The  Major  Elements  of  the  BSC. 


All  operational  BN  radio  nets  are  simulated, 
including  the  play  of  electronic  countermeasures 
(ECM).  The  command  and  staff  receive  all  inputs 
and  issue  orders  on  the  simulated  nets.  As  far 
as  the  commander  and  his  staff  are  concerned,  the 
interaction  on  the  radio  nets  and  the  type  of 

information  exchanged  are  the  same  as  would  be 

found  in  an  actual  battle. 

The  BN  commander  and  staff  do  not  see  that 

the  company  commanders,  BCE  staff  and  fi^es/air 
pe. sonnel  are  working  at  computer  stations  in  the 
control  room.  Orders  from  the  BN  commander  are 
interpreted  by  the  company  commanders  much  as 

they  would  be  in  the  field,  and  entered  into  the 
computer.  The  battle  simulation  system  moves 
units  and  determines  the  outcomes  of  conflicts. 
The  results  are  displayed  on  the  monitors,  and 
reported  to  the  BN  commander  and  staff.  Simi¬ 
larly,  fires/air,  acmin/log  and  BDE  HQ  are  played 
as  they  would  be  in  the  field. 

The  BN  actions  are  countered  by  OPFOR 
players  who  are  free  to  act  on  their  own,  con¬ 
sistent  with  the  doctrine  of  the  enemy  forces. 
OPFOR  units  in  the  simulation  can  thus  counter  BN 
actions,  and  attempt  to  take  advantage  of  any 
opportunities  offered  by  BN  player  actions. 

Battle  sounds,  ranging  from  small  arms  fire 
to  artillery,  are  provided  and  represent  an 
important  part  of  the  realistic  environment.  The 
type  of  sound,  level,  and  direction  are  all  con¬ 
trolled  from  a  recorder  room  to  reflect  the  bat¬ 
tle  situation. 

Observation  of  player  activities  for  exer¬ 
cise  control  and  eventual  player  feedback  is  pro¬ 
vided  by  a  closed-circuit  TV  (CCTV)  system  to 
maintain  a  realistic  environment.  Scenes  from 
the  TOC,  admin/log  and  jump  TOC  are  monitored  and 
recorded  in  the  recorder  room.  Provision  is  made 
for  visitors  to  observe  control  roor.i  activities 
and  view  CCTV  scenes  of  TOC  activities  from  a 
separate  visitor  room.  A  briefing  room  is  also 
provided  to  conduct  after-action  reviews.  All 
the  controllers,  players,  the  BN  commander  and 
staff  participate. 

The  BSC  concept  has  been  proven  effective  ir 
years  of  operating  CATTS  and  CAMMS,  and  ir  the 
operational  tests  of  MACE.  The  US  Army  has  suc¬ 
cessfully  applied  the  concept  to  training  BN  com- 
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manders  and  their  staffs  for  eight  years  in  the 
CATTS  operation  a*  Ft.  Leavenworth.  The  more 
recent  CAMMS  system  has  less  resolution  than 
CATTS,  but  has  shown  that  computer-assisted  BN 
level  C2  training  can  be  successfully  conducted 
on-site  via  phone  links  to  a  large  central  compu¬ 
ter.  The  MACE  BSC  more  closely  resembles  CATTS 
in  terms  of  fidelity  to  an  actual  battle  situa¬ 
tion.  The  MACE  simulation  is  an  upgrade  of  the 
CAMMS  software,  redesigned  to  fit  a  low  cost 
microcomputer  network  that  is  readily  implemented 
on-site. 

THE  MACE/MICRO-DISC  SYSTEM 
MACE--an  Improved  Version  of  CAMMS 

MACE  uses  the  conflict  algorithms  and  combat 
'resolution  data  from  CAMMS  reorganized  to  operate 
on  a  distributed  network  of  personal  microcompu¬ 
ters.  Videodiscs  were  added  to  the  system  to 
stor^  and  display  standard  military  maps  on  color 
monitors.  Programming  was  added  to  the  microcom¬ 
puters  to  generate  displays  of  unit  1ocation  and 
status  as  overlays  on  the  maps. 

MACE  can  accommodate  armor,  airborne,  air¬ 
mobile,  cavalry,  regular  and  mechanized  infantry 
maneuver  BNs,  and  their  _ombat/combat  service 
support  elements.  The  simulation  provides  real 
time  outputs  for  battle  assessments,  casualty 
reports,  ammunition  and  supply  reports,  and  pe •  - 
sonnel  reports.  The  free  play  feature  of  MACE 
provides  the  BN  commander  and  his  staff  with  a 
realistic  combat  situation  to  exercise  their  com¬ 
mand  and  control  techniques  against  an  adversary 
who  can  adapt  and  respond  to  BN  actions.  The 
scenarios  that  can  te  developed  are  bound  only  by 
the  availability  of  exercise  area  maps  on  the 
videodisc. 

Operational  Configuration 

The  MACE/Micro-Disc  System  in  the  BSC  con¬ 
trol  room  is  organized  into  five  operating  sta¬ 
tions  corresponding  as  closely  as  possible  to  BN 
operational  requirements  (Figure  3'.  There  are 
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some  constraints  due  tn  the  design  of  the  soft¬ 
ware  and  the  allocation  of  operations  to  differ¬ 
ent  microcomputer  modules. 

The  main  game  control  station  has  positions 
for  the  exercise  director,  the  chief  controller, 
the  OPFOR  team  chief,  and  the  BDE  S2  and  S3 
player/ccritrollers.  It  is  the  responsibility  of 
this  group  to  ensure  the  smooth  functioning  of 
the  simulation,  and  proper  use  of  each  of  the 
ocher  modules  of  the  system.  A  large  screen  dis¬ 
play  is  operated  from  the  main  game  control  sta¬ 
tion  to  provide  a  common  display  for  reference 
when  individual  stations  are  focusing  on  differ¬ 
ent  areas  of  the  battlefield. 

At  each  of  the  two  maneuver/confl  ict  sta¬ 
tions,  the  player/control lers  determine  friendly 
and  OPFOR  company  leve1  actions,  and  provide 
coordination  between  the  maneuver  companies, 
other-  BN  elements,  and  BN  and  BDE  headquarters. 
Friendly  company  maneuver  commands  are  determined 
by  the  company  commanders  based  on  BN  orders  and 
their  own  judgment.  The  OPFOR  player  determines 
commands  for  his  forces  in  coordination  with  his 
team  chief  at  the  main  game  cnntrol  station. 
Both  friendly  and  OPFOR  commands  a»e  entered  into 
the  station's  maneuver/conflict  microcomputer 
module.  The  controller  is  responsible  for  ensur¬ 
ing  accurate  inputs  and  for  initiating  conflict 
when  it  is  appropriate.  The  map  and  overlay  dis¬ 
play  at  this  station  supports  the  development  of 
the  command  inputs  with  detailed  unit  location, 
status  and  action  information  The  company  com¬ 
manders  at  each  station  are  assisted  in  their 
work  by  one  company  executive  officer  (XO)  or 
first  sergeant  ( 1 SG)  and  the  Fire  Support  Team 
(FIST)  member.  In  addition,  the  XO  or  1SG  pro¬ 
vides  coordination  with  BN  staff  and  with  the 
admin/log  station.  The  FIST  provides  coordina¬ 
tion  with  the  fires/air  station. 

The  admin/log  station  has  two  positions. 
One  position  is  for  the  BDE  admin/log  controller 
who  is  responsible  for  the  input  of  brigade 
responses  to  requested  support  from  battalion 
admin/log.  The  other  position  is  the  BDE  S1/S4 
player.  The  S1/S4  player  receives  status 
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reports,  and  processes  requests  for  supplies  and 
personnel  from  the  combat  trains. 

The  fires/air  station  has  an  Of FOR  player  to 
determine  enemy  artillery  and  air  actions.  The 
artillery  player  coordinates  the  requests  for 
a-tillery  support.  Requests  for  air  missions  are 
coordinated  by  the  air  player. 

Oper< tor  Station  inscription 

At  a  typical  operator  station  (Figure  4), 
there  are  two  microcomputers  and  associated  input 
and  display  peripherals.  The  data  system  micro¬ 
computer  generates  results  for  a  portion  of  the 
simulation  such  as  maneuver  or  admin/log.  The 
graphics  system  microcomputer  drives  a  videodisc 
player  and  a  graphics  generator.  The  graphics 
generator  mixes  the  map  background  video  from  the 
videodisc  player  with  symbolic  overlay  video  to 
produce  the  current  situation  map  on  the  video 
display. 


Figure  4.  MACE /Mi c ro-Di sc  Operator  Station. 


Maneuver,  admin/log,  and  fires/air  command;, 
are  entered  through  the  data  system  microcomputer 
keyboard  or  the  video  system  graphics  tablet. 
Keyboard  entries  to  the  video  system  microcompu¬ 
ter  call  up  the  map  display  and  a  joystick  pro¬ 
vides  scroll  and  zoom  control.  Information  for 
the  associated  overlay  is  entered  throuqh  the 
keyboard  or  a  graphics  tablet.  Maneuver  com¬ 
mands,  admin/log  commands,  and  fires/air  commands 
are  also  entered  through  the  keyboard  or  graphics 
tablet. 

Operation  of  a  MACE  station  is  similar  to 
CAMMS  in  terms  of  the  computer  interaction  fer 
command  input  and  text  display.  Army  personnel 
experienced  in  the  use  of  CAMMS  have  been  able  to 
operate  command  input  and  text  output  with  little 
additional  instruction.  About  five  hours  of 
training  and  oract.ce  is  sufficient  to  acquaint 
pi  ayer/control1 ors  with  the  MACE  keyboard  com¬ 
mand,  graphics  tablet  procedures,  and  joystick 
control  of  the  map  displays  and  overlays. 

The  Map  and  Overlay  Display 

A  typical  MACE/Micro-Di sc  System  display  of 
a  military  map  from  a  videodisc  and  the  associ¬ 
ated  unit  status  overlay  is  shown  in  Figure  5. 
The  operator  can  use  the  joystick  to  scroll 
across  the  map  display  and  zoom  in  on  a  section 
of  particular  interest.  The  zoom  control  permits 
selection  of  a  display  ranging  from  40  to  4  kilo- 


Fiqure  5.  MACE/ Mi cro-D i sc  Display  of  Map 
and  Ck'(  rlay. 


neters  on  a  side.  The  unit  status  overlay  is 
keyed  to  the  map  coordinates,  and  moves  with  the 
map  as  the  operator  exercises  the  scroll  and  zoom 
features . 

MACE/Micro-Di sc  System  Distributed  Architecture 

The  Micro-Disc  System  distributed  architec¬ 
ture  provides  common  use  of  mass  storage,  access 
to  all  simulation  functions  through  each  micro¬ 
computer,  graceful  degradation  in  the  event  of 
microcomputer  failure,  enhanced  continuity  of  the 
situation  display  and  ready  expansion.  In  the 
Micro-Disc  System  the  microcomputers  are  inter¬ 
connected  by  a  local  area  network  with  a  shared 
hard  disk  mass  storage  device  (Figure  6).  There 
are  five  modules,  co;  *esponding  to  the  operator 
stations  shown  in  Figure  3  and  the  organization 
of  the  simulation  software. 

The  distributed  architecture  of  MACE  pro¬ 
vides  a  number  of  advantages: 

(1)  FLEXIBILITY  -  Any  of  the  functional 
modules,  such  as  fires/air.,  may  be  run 
from  any  of  the  operator  stations,  pro¬ 
vided  that  the  appropriate  peripheral 
devices  are  attached. 

(2)  CONTINUITY  -  If  isolated  equipment 
failures  occur  during  an  exercise,  it 
is  possible  to  reconfigure  to  run  in  a 
somewhat  degraded  mode  and  avoid  sacri¬ 
ficing  time  already  invested.  For 
example,  if  the  admin/log  data  system 
microcomputer  fails,  one  of  the  two 
maneuver  stations  can  be  reconfigured 
to  perform  the  admin/log  function. 
Only  one  maneuver  station  would  then  be 
on  line,  but  all  module  types  would  be 
active. 

13)  SYNERGISM  -  Because  of  the  shared  file 
conceptual  approach  utilized  by  MACE, 
combined  battle  effects  can  be  taken 
into  account.  For  example,  movement 
orders  given  to  an  engaged  unit  will 
produce  a  rate  of  movement  which  is 
affected  by  such  factors  as  suppres¬ 
sion,  weather  and  terrain  conditions. 
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f'io'ire  6.  MACf-  Miirc-:1’^  Arc  h i  ter tore . 


The  MACE/Micrc-Disc  System  Components 

The  components  of  the  MACE/Micro-Disc  syster 
provide  proven  operational  capability,  high  reli¬ 
ability  and  low  cost.  The  microcomputers,  for 
example,  have  a  negligible  rate  of  failure  except 
in  cases  where  they  have  been  mistreated  (e.g., 
hy  inserting  or  removing  an  interface  card  before 
the  power  has  been  shut  down).  The  hard  disk, 
while  the  most  sensitive  piece  of  hardware,  has  a 
very  large  mean-time-betwee^- failures  and  ha. 
weathered  development  and  testing  very  well.  The 
video  equipment  also  has  proven  to  be  quite 
stable.  Once  adjusted,  the  grapnics  generators 
have  continued  to  operate  without  any  significant 
down  time. 

Operationally,  the  various  components  of  the 
MACE/Micro-Disc  System  have  performed  well.  The 
graphics  system  produces  clear,  medium  resolution 
symbolic  overlays  on  quite  readable  map  back¬ 
grounds,  The  graphics  tablets  allow  for  rapid 
input  of  scenario  data.  The  microcomputers  have 
demonstrated  their  ability  to  do  all  the  requi¬ 
site  processing  in  nrsr  real  time  to  allow  a 
real istic  BN  simulation. 

The  local  area  network  implementation  pro¬ 
vides  the  necessary  hardware  linkage  to  multipl> 
the  effective  computing  power  of  the  individual 
processors  by  allowing  them  to  share  access  to 
the  same  data  base. 

Microcomputer  r ugged izat ion  provides  import¬ 
ant  reliability  modifications  by  sirrtpl  i  fying  the 
connections  to  the  computer  while  limiting  access 
to  printed  circuit  boards  (Figure  7 ) .  These  rug- 
gedization  modifications  are  such  that  all  con¬ 
nections  for  peripherals  are  made  on  the  back 
panel  connectors  of  the  microcomputers.  To 
ensure  proper  connections  are  made,  each  per¬ 
ipheral  has  a  unique  connector  which  is  clearly 
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marked.  The  modifications  also  include  a  six 
ampere  power  supply  and  an  internal  computer  fan 
with  baffling  system  to  prevent  the  interior  com¬ 
ponents  from  overheating.  Flexibility,  durabil¬ 
ity  and  ease  of  use  are  greatly  enhanced  by  these 
modi fications. 

OPERATIONAL  TEST  RESULTS 

Operational  tests  of  the  MACE  maneuver  BN- 
level  training  system  were  conducted  at  Fort 
Lewis,  Washington  during  the  months  of  February 
and  March  198?.  The  tests  were  conducted  through 
the  exercise  of  five  BN  command  groups  of  the  Qth 
Infantry  Division.  Acceptance  of  the  simulation 
as  a  training  system  was  extremely  high  among  the 
partic ipants. 
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The  tests  established  that: 

(1)  it  is  possible  to  convert  the  existing 
CAMMS  program  to  operate  on  a  distri¬ 
buted  network  of  five  microcomputers; 

(2)  the  Micro-Disc  hardware  is  cost-effec¬ 
tive; 

(3)  non-ADP  (automated  data  processing) 
personnel  can  operate  the  MACE/Micro- 
Disc  System; 

(4)  no  additional  training  aides  are  needed 
to  support  BN-level  CPXs;  and 

(5)  storage  capacity  for  data  in  the  MACE./ 
Micro -Disc  System  is  sufficient. 

The  tests  also  showed  a  need  to  declutter  the 
graphics  display  and  speed  up  processing  to  avoid 
simulation  celays  when  25  or  more  units  are 
active.  The  latter  requirement  led  to  the  inves¬ 
tigation  of  a  more  powerful  "high  density  data" 
microcomputer  to  be  used  in  the  main  game  module 
in  place  of  the  original  data  system  microcompu¬ 
ter.  Improvements  to  the  graphics  software  have 
been  identified  and  will  be  incorporated  in  oper¬ 
ational  versions  of  the  MACE/Micro -Disc  System. 

BN  commanders  and  their  staffs  participating 
in  the  Ft.  Lewis  tests  rated  the  MACE/Micro-Disc 
System  higher  than  CAMMS  or  manual  CPXs.  Those 
with  CATTS  experience  rated  MACE  as  comparable  to 
the  more  complex  and  higher  cost  system. 

CONCLUSION 

The  MACE/Micro-Disc  system  has  been  shown  to 
be  effective,  easy  to  use  and  low  cost.  It  is  a 
system  that  meets  the  Army's  need  now  for 
improved  C?  training  at  the  maneuver  BN  level. 
Furthermore,  t  demonstrates  what  can  be  done 
with  rapidly  emerging  microcomputer  and  videodisc 
technologies  combined  with  distributed  processing 
concepts . 

Thv.  growth  potential  of  the  system  ensures  a 
long  and  effective  life  for  the  concept.  The 
ability  tc  substitute  more  powerful  microcompu¬ 
ters,  such  as  being  considered  for  the  Main  Game 
Module,  and  the  ability  to  network  up  to  a  total 
of  64  microcomputer  devices  provide  major  growth 
potential.  More  complex  simulations  can  be  read¬ 
ily  accommodated. 

The  operational  capability  of  the  BSC  can  be 
lurther  enhanced  by  adding  new  software  modules 
for  non-maneuver  BN  training.  This  can  include 
engineer,  medical  and  artillery  BNs. 

The  growth  potential  in  the  use  of  the  cur¬ 
rent  configuration  is  perhaps  equally  important. 
Local  needs  for  individual  or  small  group  spe¬ 
cialized  training  can  be  met  by  using  the 
Micro-Disc  System  for  other  war  games  or  training 
packages  when  it  is  not  in  use  for  MACE  exer- 
c i ses. 

Finally,  the  Army  has  an  opportunity  to  put 
a  large  number  of  systems  in  the  field.  This 
greatly  expands  the  amount  and  quality  of  train¬ 
ing  of  local  commands,  and  creates  an  enormous 
base  of  information  on  what  works  and  does  not 


work  in  the  field.  That  information,  properly 
gathered  and  analyzed,  can  shape  even  more  effec¬ 
tive  training  systems  for  the  future. 
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ABSTRACT 


In  order  to  optimize  the  capability  of  today's  ASW  equipment,  an  oper¬ 
ator  must  be  proficient  in  both  the  technical  and  operational  aspects  of 
system  employment.  Providing  adequate  training  to  personnel,  however,  is 
straining  present  resources.  An  effective  onboard  ASW  training  and  readi¬ 
ness  assessment  capability  that  uses  available'  shipboard  personnel  and 
equipment  is  one  way  of  meeting  future  training  needs.  This  paper  pro¬ 
vides  an  overview  of  an  onboard  training  device  that  is  now  installed 
aboard  SSLN  submarines.  The  various  issues  and  lessons  learned  during 
system  definition,  development  and  fleet  evaluation  will  be  discussed,  and 
the  system  approach  and  implementation  will  be  described^. 


THE  NEED 

Advances  related  to  the  Soviet  sea 
warfare  threat,  both  in  the  areas  of  in¬ 
creased  tactical  capability  and  increased 
numbers  of  vessels,  have  been  countered 
by  a  strategy  requiring  rapid  increases 
in  the  capability  of  our  antisubmarine 
warfare  (ASW)  systems.  To  ensure  that 
our  new  systems  are  effectively  used  re¬ 
quires  that  the  operators  have  an  exten¬ 
sive  technical  and  operational  under¬ 
standing  of  system  employment.  Optimum 
utilization  of  personnel  and  equipment  to 
meet  changing  requirements  requires  con¬ 
tinuous  modification  and  evaluation  of 
operating  guidelines.  Proficiency  in  all 
areas  of  maintenance,  operations  and  tac¬ 
tical  employment  must  be  maintained  at  a 
high  level  to  take  advantage  of  our  qual¬ 
itative  performance  superiority. 

The  complexity  of  the  systems  has  led 
to  longer  training  periods.  This  period 
has  aggravated  the  problem  of  optimum 
usage  of  personnel.  As  many  as  21  per¬ 
cent  of  total  Navy  personnel  are  in 
training  at  any  time.  The  lack  of  avail¬ 
ability  of  trained  people  coincides  with 
a  rapid  turnover  of  experienced  fleet 
personnel.  These  senior  people,  with 
many  years  of  practical  knowledge,  have 
been  invaluable  as  educators  and  trusted 
advisors.  These  ASW  operations  "mentors" 
in  the  past  could  provide  a  level  of  ex¬ 
perience  and  knowledge  that  could  be 
used,  especially  aboard  ship,  to  ensure 
the  continuous  proficiency  of  the  sonar 
operators. 

The  problems  associated  with  continu¬ 
ously  evolving  tactical  equipment  and 
operational  guidelines,  lack  of  qualified 
personnel,  rapid  turnover  of  experienced 
operators,  and  the  ever- expanding  train¬ 
ing  requirements,  must  be  dealt  with  to 
ensure  an  adequate  utilization  of  our  ad¬ 
vanced  system  capability. 


BACKGROUND 

Over  the  past  ten  years,  Raytheon 
Submarine  Signal  Division  has  been  in¬ 
volved  in  a  novel  approach  to  providing 
ASW  training  with  the  development  of 
shipboard  training  equipment.  Though  on¬ 
board  training  is  not  new,  the  capability 
of  providing  realistic  operational  train¬ 
ing  while  using  the  actual  shipboard  tac¬ 
tical  equipment  and  environment  has  only 
recently  been  technically  practical. 
Previously,  the  equipment  necessary  to 
adequately  perform  the  signal  processing 
functions  needed  would  have  been  coo 
large  to  be  placed  aboard  the  ship.  Such 
devices  were  only  considered  for  large 
shorebased  training  facilities.  With  the 
electronic  advances  in  the  area  of  large 
scale  integrated  circuits,  especially  the 
development  of  the  microprocessor,  a 
sophisticated  training  system  could  be 
developed  for  shipboard  use.  Initial 
development  of  the  concepts  used  today 
began  in  the  early  70s  Raytheon  devel¬ 
oped  a  system  that  could  stimulate  the 
actual  ship's  sonar  system  with  synthe¬ 
sized  acoustic  signals.  It  was  origi¬ 
nally  conceived  as  a  maintenance  aid  for 
the  AN/BQS-13  sonar  system  used  on  nucle¬ 
ar  attack  submarines.  However,  it  rapid¬ 
ly  became  obvious  that  such  a  system 
could  provide  a  unique  training  capabil¬ 
ity  of  unmatched  flexibility  and  real¬ 
ism.  By  injecting  a  synthesized  signal 
into  the  sonar  system,  appearing  to  the 
operator  as  an  actual  contact  or  target, 
it  was  possible  to  train  the  operacor 
repeatedly  and  predictably  on  his  "own" 
equipment  at  any  convenient  time.  The 
ability  of  the  system  to  provide  effec¬ 
tive  onboard  training  was  successfully 
evaluated  by  the  Navy  in  1973,  when  an 
engineering  development  model  of  the 
trainer  war  tested  (Figure  1) .  With  this 
evaluation,  a  new  alternative  to  provid¬ 
ing  and  maintaining  sonar  operator  profi¬ 
ciency  was  demonstrated. 
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Figure  1.  Preproduction  Version  of  the  AN/BQR-T4  ,  the  DS1210 


ISSUES 

The  evaluation  of  this  early  system 
as  an  onboard  operational  training  device 
uncovered  several  issues  that  have  influ¬ 
enced  the  development  of  succeeding  sys¬ 
tems;  they  are: 

■  Ease  of  operator  control  of  the 
training  system 

■  Consistency 

■  Target  realism. 

Ease  of  Operator  Control 

The  training  system  is  used  aboard 
ship,  in  manv  instances  at  sea,  by  ship¬ 
board  personnel  who  are  not  instructors. 
Under  these  circumstances  it  is  important 
that  the  control  of  the  training  device 
be  as  straightforward  as  possible.  The 
training  device  is  used  to  create  a 
training  environment.  In  order  to  make 
this  as  effective  as  possible,  there 
should  be  very  little  overhead  relative 
to  turning  on  and  setting  up  the  system. 
It  should  be  as  easy  as  possible  to  get 
into  the  training  exercise. 


Consistency 

Along  with  the  consideration  that  the 
control  of  the  training  system  be  as 
straightforward  as  possible  to  conduct  a 
training  exercise  easily,  it  is  vital 
that  the  training  environment  that  is 
created  be  as  consistent  as  possible  with 
the  normal  operational  conditions  ex¬ 
pected  by  the  sonar  operators.  This  con¬ 
sistency  is  very  important.  There  is  no 
instructor,  as  at  a  shorebased  facility, 
to  provide  the  continuity  needed  if  the 
training  situation  does  not  match  the 
true  operational  conditions.  The  realism 
provided  by  the  training  device  must  be 
sufficient  to  allow  the  student  to  per¬ 
ceive  a  situation  as  consistent  with  his 
past  training  and  operational  experi¬ 
ence.  If  not,  the  inconsistencies  will 
interfere  with  the  training  process. 
Without  an  instructor  to  explain  away 
major  inconsistencies,  the  risk  is  that 
the  training  could  be  perceived  as  not 
appropriate  and  the  training  device  might 
not  be  used  at  all. 

The  consistency  issue  applies  to  many 
aspects  of  the  training  environment,  in¬ 
cluding  turnon,  setup,  and  employment  of 


54 


-  ;  -I 


the  tactical  system,  whether  it  be  an  in¬ 
dividual  equipment  or  the  entire  sonar 
ASW  group.  Ideally,  the  training  system 
interaction  with  the  tactical  equipment 
is  transparent  so  that  no  unique  setup  of 
the  tactical  equipment  is  associated  with 
the  training  system  being  on.  All  opera¬ 
tional  modes  should  be  available  to  the 
operators  so  that  training  can  include 
turnon,  setup,  detection,  classification 
and  localization  in  a  manner  consistent 
with  what  the  operator  should  expect  when 
he  is  using  his  equipment  properly. 

Target  Realism 


In  order  to  provide  the  possibility 
of  training  for  the  large  variety  of  con¬ 
ditions  that  an  operator  is  confronted 
with  during  the  detection,  classification 
and  localization  of  a  targec,  it  is  im¬ 
portant  that  the  training  device  provide 
a  realistic  training  target.  Here  again 
the  issue  of  consistency  is  important. 
If  an  operator  is  to  be  trained  to  best 
use  his  equipment  to  extract  the  maximum 
amount  of  information,  it  is  important  to 
provide  the  capability  to  exercise  the 
operator  i:i  a  manner  as  consistent  with 
normal  conditions  as  possible.  Target 
type,  signal  content,  narrowband  and 
broadband,  ownship  and  target  maneuver¬ 
ing,  and  sonar  signal  processing  setup 
can  create  many  conditions  that  cause 
unique  presentations  at  the  operator's 
sonar  display.  By  providing  a  reasonably 
realistic  training  target,  an  operator 
can  be  taught  to  recognize  and  respond  to 
the  key  characteristics  that  allow  him  to 
best  employ  his  sonar  system. 

Particular  attention  must  be  paid  to 
areas  where  cognitive  decision-naking 
takes  place.  Subjective  analysis  of  tar¬ 
get  characteristics  as  seen  on  sonar  dis¬ 
plays  is  an  area  that  requires  repetitive 
training  to  maintain  proficiency.  By 
providing  a  realistic  target  and  allowing 
the  operator  free  play  in  the  use  of  his 
equipment,  he  can  best:  appreciate  the 
interaction  of  incorrect  equipment  set¬ 
tings  on  the  degradation  of  the  detected 
target  signal. 

Now  in  Service  Use 


These  issues  provided  the  oasic  guid¬ 
ance  in  selecting  the  particular  cystem 
approach  and  implementation  used  during 
the  development  of  the  production  version 
of  the  training  device. 

The  first  production  version  of  the 
onboard  trainer  was  the  DS1210.  The  sys¬ 
tem  has  received  approval  for  service 
use,  is  installed  on  all  fleet  ballistic 
missile  submarines  and  has  been  assigned 
the  military  designation  AN/B0R-T4 . 

Over  many  patrols,  combat  teams  have 
been  enthusiastic  about  the  high  degree 
of  realism  which  the  trainer  makes  possi¬ 
ble.  The  concepts  originated  to  evaluate 
the  first  training  system  have  been  ex¬ 


panded  to  a  sophisticated  training  pro¬ 
gram  known  as  SOTAP:  Sonar  Operational 
Training  and  Assessment  Program.  SOTAP 
is  a  Navy-managed  program  used  to  coordi¬ 
nate  sonar  operator  training  to  ensure 
optimum  use  of  both  the  shnrebased  train¬ 
ing  facilities  and  the  onboard  training 
equipment. 

Front-End  Stimulation 


The  success  of  the  onboard  training 
system  in  providing  an  effective  capabil¬ 
ity  is  due  to  the  design  approach  chosen 
and  its  implementation.  The  DS1200 
series  trainers  are  "stimulation”  type 
systems  rather  than  "simulation"  sys¬ 
tems.  The  primary  functional  difference 
is  that  the  stimulator  uses  as  much  of 
the  tactical  sonar  equipment  as  possible 
where  as  the  simulator  type  system  "simu¬ 
lates"  mathematically  as  much  of  the 
equipment  as  possible.  With  the  stimula¬ 
tor,  the  goal  is  to  model  the  threat  and 
acoustic  phenomenon  as  realistically  as 
practical,  synthesize  analogous  signals, 
and  stimulate  the  sonar  with  these  sig¬ 
nals.  The  fact  that  the  actual  sonar 
system  equipment  is  used  to  process  the 
signals  provides  a  very  realistic  presen¬ 
tation  to  the  sonar  operator.  It  is  this 
realism  that  allows  training  to  take 
place  on  many  levels,  from  simple  proce¬ 
dural  training  to  levels  that  require  an 
interpretive  assessment  on  the  part  of 
the  sonar  operator  to  ensure  proper  util¬ 
ization  of  the  sonar  equipment.  To  date, 
the  degree  of  realism  attained  with  stim¬ 
ulation  has  not  been  reached  using  the 
simulation  approach.  The  complexities  of 
the  ocean  medium  and  the  difficulty  of 
modeling  the  sonar  system  processing  have 
made  it  impractical  when  a  high  degree  of 
fidelity  is  needed. 

The  signals  generated  by  the  training 
system  are  injected  into  the  sonar  system 
as  close  to  the  "front-end"  as  possible 
(See  Figure  2.)  This  allows  all  sonar 


Figure  2.  Flow  of  Injected  Synthesized 
S  ignals 
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processing  to  continue  during  the  train¬ 
ing  exercise.  It  also  does  not  interfere 
with  normal  operator;  functions.  The 
operator  does  not  have  to  configure  the 
sonar  into  a  "training"  mode;  therefore, 
the  operator  is  trained  in  an  environment 
consistent  with  the  normal  operation  of 
the  tactical  equipment.  Added  advantages 
to  front-end  stimulation  are  that  test 
signals  can  be  injected  into  the  sonar 
system  for  maintenance  purposes,  any 
auxiliary  processing  equipment  used  with 
the  sonar  can  also  be  stimulated,  and  the 
trainer  system  is  relatively  immune  to 
changes  that  may  take  place  to  improve 
the  processing  capability  of  the  sonar 
system . 

The  signals  injected  into  the  sonar 
are  synthesized  replicas  of  signals  nor¬ 
mally  expected  from  the  transducer  array 
assembly.  The  training  system  has  to 
have  within  it  a  "world  of  its  own."  The 
training  system  provides  its  operator  the 
capability  of  selecting  various  target- 
and  ocean  types.  The  selected  target 
vessel  can  then  be  maneuvered  as  though 
the  trainer  operator,  using  the  instruc¬ 
tor's  console,  were  in  fact  the  target 
vessel  operator.  (Figure  3.) 


Within  the  equipment,  Hie  maneuvers 
of  the  target  and  ownship  are  monitored 
and  maintained.  The  radiated  target  ves¬ 
sel  signals  are  synthesized  to  reflect 
the  varied  cha rac ter i s t  i  c  of  machinery, 
propulsion,  propeller  and  hydrodynamic 
sound  sources.  These  radiated  signals 
are  modified  to  reflect  the  complex  na¬ 
ture  of  propagation  through  the  selected 
ocean.  An  array  simulator  then  processes 
the  signals  to  reflect  the  conversion  of 
the  synthesized  sound  pressure  signal  by 
the  sonar  transducer  array  assembly. 
These  signals  are  then  injected  into  the 
sonar  to  be  processed  as  actual  acoustic 
data  (see  Figure  4). 

Of  particular  interest  is  the  complex 
nature  of  the  radiated  spectrum  from  the 
target  vessel.  In  order  to  realistically 
simulate  these  signals,  the  training  sys¬ 
tem  must  be  able  to  recreate  the  "in¬ 
sides"  of  the  selected  target  vessel. 

The  sounds  from  auxiliary  equipment, 
diesel-engine  firing,  the  whine  of  gears, 
the  hum  of  generators,  propeller  cavita¬ 
tion,  and  water  flowing  over  the  hull  are 
characteristic  sounds  that  provide  clues 
to  the  sonar  operator  of  the  vessel 


Figure  i.  Trainer  Control  Unit  of  AN/B0R-T4  as  Installed  in  a 
Fleet  Ballistic  Missi le  Submarine 
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SIGNAL 
SPECTRUM 
AS  MODIFIED 
BY  PROPAGATION 


S 


Figure  4.  Trainer  Simulated  World 


type.  These  various  characteristics  are 
synthesized  by  a  microprocessor-con¬ 
trolled  digital  signal  generator  that  is 
programmed  to  model  various  target  types 
(see  Figure  5) . 

The  system  approach  allows  for  ready 
adaption  to  different  sonar  systems.  The 
target  signal  synthesizers  and  the  ocean 
processing  have  been  designed  to  approxi¬ 
mately  model  the  effects  of  the  respec¬ 
tive  phenomena,  independent  of  the  tac¬ 
tical  system  to  be  stimulated.  The 
training  system’s  array  simulator  can  be 
programmed  to  model  various  geometric 
configurations  of  sonar  system  sensor 
arrays.  It  is  the  function  of  the  array 
simulator  to  provide  the  unique  proces¬ 
sing  needed  t  adapt  to  various  sonar 
system  characteristics 

The  system  concept  described  has  been 
successfully  applied  to  submarine  and 
surface  ship  soaor  systems.  Newer  ver¬ 
sions  of  the  system  allow  for  multiple 
targets  to  be  synthesized  and  controlled 
simultaneously.  Evolutions  of  system 
capabilities  havi  led  to  ownship  noise 
and  ocean  nois**  synthesis  to  allow  real¬ 
istic  dockside  training  to  be  conducted. 


Ownship  motion  is  simulated  so  that, 
dockside  or  during  transit  periods,  the 
training  can  allow  own  ship  to  prosecute 
the  training  target  without  interfering 
with  normal  evolutions.  Interfaces  have 
been  provided  with  the  fire  control  sys¬ 
tem  aboard  ship  so  that  ar.  entire  fire 
control,  sonar  team  exercise  can  be  run. 
In  fact,  any  combat  system  function  that 
is  provided  tactical  information  from  tne 
sonar  can  be  exercised  in  a  training 
situation. 


THE  TUTURE 

The  future  of  the  onboard  operational 
trainer  can  lead  to  a  means  of  providing 
a  source  of  information  that  previously 
was  supplied  by  key  experienced  personnel 
or  mentors.  This  capabili'  '  can  be  pro¬ 
vided  to  the  ASW  team  or  in  fact,  any 
sensor  or  combat  team.  A  trainer  can  be 
configured  for  each  major  sensor  group 
and  it  can  become  a  source  of  operability 
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Figure  5.  Microprocessor -Controlled 
Synthesizer 


information  providing  not  only  a  training 
capability  but  a  library  of  operation 
guidelines,  a  source  of  maintenance  in¬ 
formation,  and  also  an  interactive  means 
of  managing  operations  for  each  team. 
Basically,  the  system  can  become  a  perma¬ 
nent  source  and  focus  of  information 
storage  that  can  aid  with  the  operations, 
maintenance,  training  and  readiness  for 
the  sensor  team:  a  source  that  is  con¬ 
sistent  from  ship  to  ship  and  a  permanent 
complement  to  the  ship's  force. 
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ABSTPACT 

The  United  States  Army  has  achieved  a  significant  advantage  over  the  Warsaw  Pact 
forces  in  the  area  of  tank  gunnery  under  conditions  of  darkness,  smoke,  fog,  and  rain. 
This  advantage  has  been  gained  through  the  development  of  the  tank  thermal  sight.  The 
ambient  temperature,  under  virtually  all  environmental  conditions,  including  smoke  and 
darkness.  Since  the  Soviets  are  particularly  adept  at  fighting  under  cover  of  smoke, 
the  ability  of  U.S.  gunners  to  acquire  a  high  skill  level  in  the  use  of  the  thermal 
sight  is  imperative  to  allow  rapid  acquisition  and  identif ication  of  enemy  vehicles  and 
personnel . 

Gunnery  training  under  adverse  conditions  of  darkness,  smoke,  fog,  and  rain  are  r.ow 
being  conducted  with  tne  aid  of  a  thermal  signature  target  developed  through  PM-TRADE. 
The  background  leading  to  the  development  of  a  Thermal  Signature  Target  as  well  as  a  de¬ 
tailed  description  of  a  modular,  full  scale  tank  Thermal  target  system  is  presented. 


INTRODUCTION  AND  BACKGROUND 

In  anticipation  of  fieldina  several  weapon 
systems  which  employed  thermal  sights,  the  Army 
issued  a  requirement  for  the  development  of  tar¬ 
gets  which  could  be  used  in  training  gunners 
associated  with  those  weapon  systems.  The 
requirements  at  that  time  were  rather  sketchy 
since  such  training  had  never  been  conducted.  The 
first  thought  was  to  require  targets  which  presen¬ 
ted  art  exact  thermal  signature  of  the  vehicle  or 
subject  being  represented .  Details  of  road  wheels, 
tracks,  engine  box  and  exhaust,  turret  and  gun 
tube  were  specified  as  distinguishing  features. 
Although  the  cost  of  such  a  target  would  be  rela¬ 
tively  high,  it  was  felt  that  target  recognition 
.Mid  identification  were  significant  elements  of 
training. 

Of  concern  to  the  Army  was  that  tne  targets 
be  low  cost,  have  low  maintenance  requirements, 
and  that  installation  could  be  completed  quickly. 

In  addition,  the  Army  was  particularly  concerned 
about  power  requirements  for  the  thermal  targets. 

A  thermal  target  which  required  no  external  power 
supplies  such  as  batteries,  generators  or  commer¬ 
cial  power  was  the  Army's  goal. 

During  the  early  development  of  the  thermal 
target,  several  candidate  targets  were  presented 
to  the  Army  by  various  government  and  commercial 
agencies.  The  forerunner  in  the  therma1  target 
area  was  NTCC,  Orlando.  (1)  Only  one  target 
which  was  demonstrated  required  no  external  power 
supply 

The  initial  technique  selected  by  the  Army  to 
fill  the  requirements  for  a  thermal  signature 
target  involved  a  chemical  heating  process.  The 


chemical  source  was  provider  in  the  form  of  a  salt 
which,  when  combined  with  water,  generated  heat. 

Initial  tests  appeared  to  be  successful;  (2) 
however,  subsequent  evaluations  of  the  chemical 
her. ting  bag  approach  revealed  several  drawbacks, 
such  as  poor  signature  in  cold  weather;  short 
duration  of  the  signature  before  requiring  a  water 
refill;  and  the  overall  effort  was  very  labor  in¬ 
tensive  . 

The  immediate  requirement  for  a  target,  which 
required  no  external  power  supply  was  dropped  in 
favor  of  a  target  which  could  be  powered  by  a  12 
volt  battery.  That  target  system  was  developed  by 
TV  I  Energy  Corporation  of  Beltsville,  Maryland. 

The  TV  I  target  is  not  considered  to  be  the  ulti¬ 
mate  solution  to  the  thermal  target  problem;  how¬ 
ever,  it  nas  been  demonstrated  to  be  a  viable 
concept. 

SYSTEM  DESCRIPTION 

T ar oe t  Module 

The  TVI  thermal  target  ’ s  made  of  modules 
wnich  can  be  arranged  ir.  any  order  to  form  the  de¬ 
sired  configuration.  Indi  v  idi*  il  modules  include 
track  sections,  turrets,  hull  sections  and  road 
wheels.  A  typical  target  sys.em  is  depicted  in 
Figure  1.  This  figure  shows  the  standard  target 
lifting  mechanism  with  a  NATO  standard  plywood 
target.  Attached  to  the  plywood  target  is  the  TVI 
thermal  signature  t  rget.  The  conf iauration  snown 
is  *hat  of  a  Russian  T62  tar.k  as  viewed  from  the 
front.  Modules  can  be  stapled  or  tacked  to  the 
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plywood  target ,  F  inure  2  shows  the  arrangement 
0 ;  thermal  modules  and  also  shows  the  Interconnec¬ 
tion  of  wires  to  the  main  target  harness.  Modules 
can  be  produced  in  basically  any  shape  and  size 
within  limits.  The  various  arrangements  which 
are  presently  in  use  .are  shown  in  figures  3  and  4. 

There  is  a  definite  advantage  to  forming  the 
thermal  target  with  nodular  units  rather  than  us¬ 
ing  a  one  piece  target.  It  has  been  found  that 
the  majority  of  rounds  strike  the  target  at  the 
center  of  mass;  i.e.,  the  engine  box  or  the  turret 
sections.  Track  sections  are  only  occasionally 
hit.  Therefore,  it  is  necessary  to  replace  only 
turret  and  the  engine  box  modules  normally,  and, 
less  frequently,  the  track  modules. 

With  the  limited  data  available  to  date,  it 
is  estimated  that  approximately  ten  105rrm  rounds 
can  penetrate  a  target  module  (approximately  50 
per  target)  without  severely  deteriorating  the 
image  quality.  This  assumes  that  a  round  does  not 
sever  the  electrical  connections  on  that  module.''' 
However  it  is  possible  to  repair  the  electrical 
connections  with  a  repair  kit  which  is  available. 
The  limiting  factor  with  the  thermal  targets  seems 
to  be  a  function  of  the  survivability  of  the  ply¬ 
wood  to  which  the  target  is  mounted. 

A  typical  module  is  shown  in  Figure  5.  The 
electro-conductive  coating  inside  the  flexible 
heating  element  operates  as  a  wide  area  resistor. 
The  actual  resistance  of  each  target  module  is  a 
fixed  value,  with  the  resistance  for  each  type  of 
target  module  (i.e.,  track,  road  wheel,  etc.)  con¬ 
trolled  to  a  relatively  tight  tolerance  in  manu¬ 
facture. 

When  an  electrical  potential  is  placed  across 
the  coating,  a  current  flows  and  power  is  dissi¬ 
pated.  Thus  certain  target  modules,  depending 
on  their  predetermined  resistance  values,  will 
appear  cooler  than  ethers  with  the  same  supply 
voltage.  This  allows  a  complete  thermal  taryet  to 
show  a  pattern  of  hotter  and  cooler  target  modules, 
without  separate  controls,  while  running  off  a 
single  power  supply.  This  provides  the  gunners 
with  the  indication  that  some  of  a  vehicle's  ther¬ 
mal  "cues"  are  more  intense  than  others  (e.g., 
that  an  engine  compartment  is  hotter  than  a  tur¬ 
ret  )  . 

The  electrical  current  flow  to  the  conductive 
coating  is  carried  by  the  copper  busbars.  These 
busbars  are  located  at  opposite  edges  of  the 
coating  in  a  manner  that  promotes  electrical  cur- 
tent  flow  throughout  the  area  of  the  conductive 
coating. 

To  ensure  reliable  and  continued  target  opera¬ 
tion  even  after  live  fire  damage,  the  busbars  in 
these  target  modules  have  power  supply  connections 
at  both  ends.  Servering  the  busbar  at  any  point 
only  results  in  the  loss  of  the  signature  from  the 
area  immediately  around  the  hole. 

Having  redundant  circuitry  also  means  that 
the  electrical  connections  to  such  a  target  module 
must  be  keyed  for  polarity.  This  is  to  ensure 
that  both  ends  of  the  same  busbar  are  connected  to 
the  same  side  of  the  power  supply.  Failure  to  do 
so  could  create  a  short  circuit  condition,  poten¬ 
tially  damaging  to  the  power  supply  and  control 


modules . 

Heating  element  performance  is  enhanced  by 
the  presence  uf  an  insulating  layer  bonded  to  the 
rear  of. the  target  module.  This  layer  of  flexible 
insulation  or  styrofoam,  depending  upon  the  type 
of  target  module,  thermally  isolates  the  heating 
element  from  the  cold  plywood  backing,  minimizing 
energy  loss  by  conduction  to  the  plywood. 

Power  Supp 1 y 

Although  the  Army  continues  to  search  for  a 
target  which  will  qenerate  a  thermal  image  without 
the  need  for  an  external  power  supply,  the  present 
target  system  does  require  power. 

Since  manv  of  the  ranges  used  by  the  Army  are 
not  equipped  with  comnercial  power,  battery  power 
is  used  extensively.  And  in  keeping  with  this 
approach,  batteries  were  used  as  the  source  of 
power  for  the  present  thennal  targets.  However, 
several  problems  appear  when  batteries  are  used  to 
power  a  heat  producing  element  such  as  the  thermal 
target.  The  present  targets  generate  approximately 
10  watts  per  square  foot  of  target  area.  Using  a 
full  size  tank  target,  a  standard  12  volt  battery 
will  dissipate  very  quickly  if  the  targets  are 
powered  continuously.  To  alleviate  this  problem, 
a  control  mechanism  was  devised  to  provide  power 
to  the  target  only  when  the  tank  crews  were  in  the 
ready-to-f '•  re  mode. 

Since  the  targets  are  mounted  on  a  lifting 
mechanism,  target  acquisition  by  the  gunners  occur¬ 
red  only  when  the  targets  are  in  a  vertical  posi¬ 
tion.  If  the  target  is  not  hit  within  45  seconds, 
the  lifting  mechanism  when  in  the  automatic  mode, 
drops  the  target  to  a  horizontal  position.  In  that 
position  there  is  no  need  to  generate  a  thermal 
image;  therefore,  power  can  be  turned  off.  By 
placing  a  mercury  switch  on  the  lifting  mechanism, 
the  power  to  the  target  can  be  turned  on  as  the 
lifting  mechanism  beqins  to  raise  the  target. 

The  lifting  time  for  the  target  mechanism  is 
approximately  6  seconds.  It  was  found  that  this 
time  was  not  long  enough  to  allow  the  target  to 
fully  heat  when  powered  by  12  volts.  To  reduce 
the  heating  time,  a  battery  inverter  was  included 
in  the  system.  The  inverter  converts  12  volts  to 
110  volts.  In  addition,  timing  circuits  were 
added  to  limit  the  time  that  power  is  provided  to 
the  target  for  each  cycle.  The  time  limit  fea¬ 
ture  makes  it  possible  to  leave  the  target  in  the 
vertical  position  when  the  lifting  device  is  in 
the  manual  mode  without  dissipating  the  battery.  It 
was  also  recommended  that  the  thermal  signature  be 
presented  no  more  frequently  than  every  10  minutes. 
During  this  time,  the  battery  can  recover  to  a  cer¬ 
tain  extent.  The  10  minutes  between  cycles  also 
more  closely  matches  the  training  scenario. 

With  the  aid  of  the  inverter,  or  power  control 
module,  and  a  few  new  procedures,  it  was  possible 
to  increase  the  time  between  battery  chargings  to 
8  hours  with  normal  use.  The  power  control  module 
is  mounted  at  the  base  of  the  target  or  on  the 
target  lifting  mechanism  as  shown  in  Figure  1. 


''get  Modules 
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Figure  4  -  Thermal  Target  Arrangements 
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Thermal  Target  Module 


Alternate  Pen ;er  Sources 

Battery  power  is  perhaps  the  least  efficient 
method  of  powering  the  thermal  targets;  however, 
the  technology  associated  with  their  hanaling  and 
maintenance  is  minimal.  Therefore,  batteries  have 
gained  widespread  acceptance.  On  the  other  hand, 
gasoline  powered  generators  are  a  considerably 
more  efficient  means  of  providing  power.  Due  to  a 
long  history  of  being  a  maintenance  nightmare, 
generators  are  generally  avoided  in  areas  where 
skilled  repairmen  are  not  readily  available.  The 
obvious  solution  would  be  to  use  commercial  power. 
The  installation  of  commercial  power  has  the  conno¬ 
tation  of  permanency,  which  means  the  ranges  are 
no  longer  flexible  and  portable. 

Despite  the  fact  that  all  of  the  power  sources 
described  above  have  some  disadvantages,  the  ther¬ 
mal  target  is  not  limited  to  any  one  source  of  pow¬ 
er.  The  targets  function  equally  well  with  battery 
power,  generator  power,  or  commercial  power. 

Thermal  Target  Training  Environment 

What  will  the  trainee  expect  to  see  through 
his  thermal  sight?  The  answer  is  that  the  image  is 
strictly  a  function  of  the  thermal  sight  design. 
Since  the  image  presented  to  the  gunner  is  a  video 
reproduction  of  data  gathered  by  the  thermal  sens¬ 
ing  devices,  image  quality  and  resolution  vary  con¬ 
siderably  from  sight  to  sight.  A  general  represen¬ 
tation  of  a  thermal  image  of  a  tank  at  night  is 
shown  in  Figure  6. 

However,  night  sighting  is  only  one  condition 
under  which  the  thermal  sight  is  a  very  useful 
acquisition  guide.  The  Warsaw  Pact  armored  units 
are  particularly  adept  at  fighting  under  cover  of 
smoke.  As  shown  in  Figures  7,  8,  and  9,  the  battle¬ 
field  can  be  cluttered  with  smoke  which  obscures 
the  enemy,  and  by  fires  which  can  provide  false 
thermal  images  to  U.S.  gunners.  Therefore,  it  is 
essential  that  gunners  receive  adequate  training  in 
target  recognition  and  identification  under  all 
conditions  cf  nighttime,  rain  and  smoke.  Figure  9 
shows  a  Soviet  tank  partially  obscured  by  smoke 
cover.  Figure  10  shows  the  same  tank  as  viewed 
through  the  thermal  sight.  In  addition  to  smoke 
cover,  trees  and  brush  are  used  to  obscure  enemy 
vehicles.  Figure  11  shows  a  tank  in  partial  defil¬ 
ade,  and  Figure  12  shows  the  same  vehicle  through 
the  thermal  sight.  Again,  these  photos  demonstr¬ 
ate  the  capability  of  the  thermal  sight. 

To  be  able  to  distinguish  the  targets  from 
the  background  requires  that  the  targets  used  for 
training  adequately  define  the  target.  A  simple 
hot  spot  or  blob  target  may  not  allow  the  gunner 
to  recognize  a  target  because  of  its  similarity 
to  the  background,  or  battlefield  conditions.  A 
fire  orr  the  battlefield,  for  instance,  will  look 
tne  same  as  a  hot  spot  target  through  the  thermal 
sight.  Therefore,  negative  training  may  be  pro¬ 
vided  if  hot  spot  targets,  rather  than  well 
defined  targets,  are  used  during  training. 

Figure  13  shows  a  TVI  thermal  signature  target 
through  the  M60A3  thermal  sight.  The  target  repre¬ 
sents  a  front  view  of  the  T62  tank.  Without  proper 
training  in  target  recognition,  if  would  be  diffi¬ 
cult  to  distinguish  the  target  from  the  background. 


The  problem  further  increases  as  the  range  to  the 
target  increases. 

Future  Potential 

Each  new  vehicle  or  weapon  system  which  em¬ 
ploys  a  thermal  sight  presents  a  new  training 
problem.  In  addition  to  simulating  vehicles,  indi¬ 
vidual  personnel  and  squads  must  also  be  simulated 
for  training  purposes.  Targets  which  can  provide 
full  scale  and  reduced  scale  signatures,  two  dimen¬ 
sional  and  three  dimensional  views  and  be  moving  or 
stationary  must  be  addressed.  In  addition,  there 
must  be  a  continuing  effort  to  develop  a  target 
which  requires  no  external  power  supply  to  produce 
a  thermal  signature.  At  the  present  time,  this 
task  appears  to  be  beyond  the  preseni  state-of-the- 
art.  However,  emerging  technology  may  provide  a 
solution  to  the  problem. 

CONCLUSION 

The  development  of  thermal  signature  targets 
through  PM-TRADE  has  provided  the  Army  with  a  sim¬ 
ple,  reliable  and  effective  tool  for  training  gun¬ 
ners  to  use  the  thermal  signt.  Efforts  will  con¬ 
tinue  to  develop  an  improved  target  with  particular 
emphasis  on  power  requirements.  The  ideal  situation 
would  be  to  have  a  thermal  target  that  required  no 
power.  However,  unless  a  major  breakthrough  occurs, 
some  form  of  power  will  be  required  to  generate  a 
thermal  signature.  The  object  of  future  efforts 
will  be  to  reduce  the  power  requirements  to  the 
lowest  level . 

Even  as  an  interim  solution,  the  TVI  thermal 
signature  target  is  a  vast  improvement  over  pre¬ 
vious,  antiquated  techniques  such  as  heating  tar¬ 
gets  with  charcocl  fires  or  smudge  pots.  Charac¬ 
teristics  such  as  quick  installation,  convenience 
in  handling,  low  cost,  modular  form,  and  the 
ability  to  withstand  several  hits  from  tank  main 
gun  rounds  make  the  TVI  thermal  signature  target 
an  extremely  viable  solution  to  the  thermal  signa¬ 
ture  target  problem. 
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Figure  6  -  Night  Sight  Thermal  Image 
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ABSTRACT 


Fc-r  years  we  have  looked  at  the  pilot,  end  (in  our  infinite  wisdom) ,  have  decided  among  ourselves  that 
Lne  more  a  simulator  looks,  feels  and  smells  like  an  aircraft  the  more  capable  a  training  device  il 
will  be.  Granted,  we  have  made  great  leaps  forward  in  computer,  visual  and  motion  system 
technology,  and  convinced  ourselves  and  many  others  of  the  great  future  of  aircrew  training  devices. 
However ,  somewhere  along  the  way  to  selling  ourselves  and  the  rest  of  the  world,  we  forgot  to  convince 
the  pilot.  "This  paper  takes  a  figurative  walk  through  the  last  20  years  in  the  simulator  world.  It 
looks,"back  through  the  microscope*  from  the  pilot’s  point  of  view.  The  accusation  is  that  (despite 
our  good  intentions)  we  have  1)  overestimated  the  simulator's  capabilties;  2)  failed  to  plan 
adequately  for  its  use;  3)  overemphasized  fidelity  in  the  plage  of  training  capability;  and  4) 
overcomplicated  these  devices  by  trying  to  incQrporate  too  many  '’whistles  and  bells. n-  In  doing  so 
we  have  developed  pilots  who  aren't  fighting  to f f ly ^simulators.  The  basic  recommendation  is  that 
we  as  developers,  buyers,  and  managers  of  aircrew  training  programs  might  better  serve  the  pilot's 
and  our  needs  if  we  did  a  better  job  looking  at  the  simulator  as  part  of  a  total  training  program 
rather  than  as  an  end  in  itself.,. 


The  scene:  a  Tactical  Air  Command  Base  somewhere 
in  the  southwest.  The  new  mission  simulator  has  been 
in  operation  for  a  month.  The  building  is  new  and 
beautiful  with  efficient  air  conditioning,  plush 
carpets,  ard  comfortable  briefing  roans.  It's  been 
said  that  this  mission  simulator  will  save  lives, 
fu  >1,  time  and  money.  It  has  the  latest  technologies 
in<  orporated  into  the  student  and  instructor 
ota  ions:  motion,  visual,  electronic  warfare,  voice 
nask.ng,  automated  instruction,  and  a  host  of  other 
goodie',  As  part  of  a  building  block  approach, 
additic.'fll  sophisticated  technologies  will  be 
incorporated  within  the  next  10  years. 

Still,  the  pilots  avoid  the  facility  like  the 
plague.  Getting  into  the  simulator  ranks  in  the 
desired  activity  list  somewhere  just  oelow  child 
support,  alimony,  and  knee  surgery.  Having  a  session 
in  the  simulator  is  sanething  pilots  avoid  ir  possible 
and  don't  admit  tc  if  forced  to. 

Hw  can  this  be?  Five  years  and  $100,000;000 
has  been  spent  to  develop,  produce  and  field  this 
simulator.  The  best  minds  of  the  developers, 
proAjcers  and  users  have  supposedly  been  put  together 
in  the  effort  to  bring  these  simulators  tn  use. 

It  really  seems  that  something's  inoongruent  has 
happened.  I  remember  my  first  experiences  with 
simulators  with  a  great  fondness  and  respect .  I  had  a 
little  simulator  time  during  pilot  training,  but  my 
first  real  experience  was  in  my  first  assignment 
flying  Aeromedical  Evaciatior.  in  the  C-131  (Convair 
240,  T-29) . 

As  I  understood  it  then,  the  simulator  had  been 
built  in  the  e*arly  fifties  and  was  jondemned  in  the 
late  fifties.  We  were  using  it  in  the  late  sixties 
because  it  was  the  only  one  that  had  ever  been  produced 
and  we  nad  to  hrve  sanething.  Tc  make  matters  worse 
it  was  designed  for  the  Oonvair  240 .  The  Convair  240 
and  the  nine  models  of  the  T-29  and  C-131  were 
basically  the  same  aircraft.  However,  there  were 


just  enough  differences  (primarily  in  the  electrical 
system)  to  make  trying  to  teach  all  models  in  the  same 
simulator  an  interesting  experience  at  best.  Talk 
about  "this  doesn't  really  fly  like  the  airplane": 
This  simulator  made  nonfidelity  an  art  form! 

It's  worth  a  few  minutes  to  digress  and  describe 
this  simulator.  It  had  what  I  would  call  a  first 
generation  visual  system:  frosted  windows  with  a 
rneostat  to  turn  the  lights  up  and  down.  This  way  you 
could  simulate  flying  in  clouds  (lights  down) ,  heavy 
clouds  (lights  further  down),  thunderstorms  (lights 
off) ,  lightning  (lights  off  with  strobe  lights  going 
on  and  off) ,  no  clouds  (lights  full  on) ,  or  breaking 
through  a  ragqed  cloud  layer  (lights  on  and  off) .  As 
an  added  bonus  there  was  a  visual  check  for  an  engine 
fire.  If  there  was  a  "real"  fire,  as  cpdof^  tz 
faulty  instrument  indications,  ♦he  instructor  could 
turn  on  a  red  el.—o  tree  light  that  flashed  }ust 
^uoiue  the  side  frosted  window  of  the  engine  on  file. 
Eat  your  heart  out  visual  engineers! 

This  simulator  was  not  lacking  in  aural  cues 
either.  The  engine  sound  track  resembled  one  of 
those  World  War  II  movies  when  300  B~17's  are  flying 
over  Potsdam.  Changes  in  power  were  merely  reflected 
by  changes  in  volume.  The  crash  noise  was  a  classic. 
I'm  just  sure  it  originated  from  an  Abbot  and  Costello 
movie.  About  the  only  thing  it  needed  to  complete  the 
vaudeville  inage  was  a  scream  at  the  end  and  the  sound 
oi  one  breaking  glass.  My  favorite  aural  cue  was  ice 
on  the  propellers.  Now  it's  tough  to  9ee  ice  on  the 
propellers  even  in  the  airplane.  Your  only  clue  is 
usually  the  sound  of  the  ice  shedding  and  hitting  the 
fuselage.  In  the  simulator  this  was  simulated  with  a 
small  axle  parallel  to  the  outside  of  the  simulator. 
Attached  to  the  axle  were  varying  lengths  of  spring 
wire  with  various  size  steel  balls  at  the  ends.  As 
the  axle  rotated  the  balls  would  spring  back  and  hit 
the  fuselage  at  irregular  intervals.  I  never  had 
real  prop  ice,  but  I  was  sure,  that  if  I  ever  did,  it 
would  sound  just  like  that. 
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simulator  was  ahead  of  its  time.  That  was  in  the  area 
of  nasal  cues.  I  have  seen  simulators  that  people 
said  "stunk",  but,  to  this  day  I  have  yet  to  see  a 
simulator  with  nasal  cues.  This  particular  feature 
was  one  of  my  favorites,  and  it  was  used  in  conjunction 
with  simulating  an  electrical  fire.  No  fancy 
keyboards  or  CRTs  existed  to  input  this  emergency. 
The  procedure  was  for  an  instructor  to  take  a  piece  of 
insulated  wire  (provided)  and  clamp  it  between  two 
terminals  next  to  the  intake  duct  for  the  cockpit  air 
conditioning  system.  Current  was  then  applied  to  the 
terminals  and  the  resulting  short  circuit  burned  the 
insulation  off  the  wire.  The  inevitable  smoke  flowed 
into  the  cockpit  with  the  trainees.  Now  that's 
imagination  and  realism  all  rolled  up  into  one. 
Which  br’ngK  'in  th«  subject  of  the  instructor 
station. . . 

The  instructor  station  was  really  something. 
As  I  said  before  no  fancy  keyboards  or  CRTs  existed. 
Neither  was  there  auto  demo,  graded  maneuvers  or 
programmed  emergencies.  The  instructor  had  before 
him  approximately  500  marked  swithches  and  rheostats 
on  an  electrical  panel.  Typically  he  was  busier  in 
the  back  than  the  guys  up  front  controlling  each 
maneuver  and  malfunction  with  a  number  of  individual 
controls . 

What  about  fidelity?  Now  there's  a  laugh.  The 
simulator  compared  to  the  Convair  like  a  Porsche  does 
to  a  Peterbilt.  You  fought  the  simulator  all  the 
time.  Talk  about  over  control;  that's  what  you  did  ii 
the  simulator.  Holding  a  heading  was  like  balancing 
on  a  beach  ball .  All  nine  models  of  the  T-29  and  C-131 
were  taught  in  the  simulator  so  the  instrument  panel 
wasn’t  correct  for  anyone.  The  electrical  system  in 
the  Convair  was  the  most  crucial  and  the  most 
difficult  system  to  learn.  In  addition,  the 
simulator  didn't  have  all  the  instruments  in  the  right 
place  for  any  of  the  models.  Talk  about  antiquity  and 
negative  state  of  the  art!  it  was  all  there  in  our 
simulator . 

Wh^t  you  may  be  expecting  to  hear  is  hew  much  we 
hated  the  simulator,  and,  with  all  the  improvements 
since  then,  why  there  isn't  any  reason  pilots  should 
feel  the  way  they  do  about  simulators.  Actually  what 
I'd  like  to  say  is  that  we  loved  it  and  there  are  some 
very  good  reasons  wny  pilots  are  not  killing 
themselves  for  the  opportunity  to  fly  simulators. 

The  Convair  simulator  was  great!  I  always 
looked  forward  to  a  week  at  the  simulator ;  even  twice  a 
year;  even  on  my  tenth  trip.  Why?  Because  I  learned 
something  and  what  I  learned  was  meaningful.  Well 
you  ask,  whit  was  so  wonderful  about  that  particular 
simulator? 

It  had  a  number  of  things  going  for  it  that  more 
than  compensated  for  the  lack  of  technology.  Many  of 
these  things  are  not  present  in  our  simulator  programs 
today.  First,  the  emphasis  was  that  the  simulator 
was  part  of  the  training  program.  Now  this  may  seori 
obvious,  but  in  many  cases  it  is  not.  Our  week  at 
Scott  was  not  one  simulator  ride  after  another.  The 
simulator  was  merely  a  tool  used  in  a  refresher  course 
which  emphasized  systan  operation  and  emergency 
procedures.  Simulator  rides  were  used  to 
demonstrate  principles  learned  in  class,  ar.phasize 
systems  lessons,  demonstrate  malfunctions  and  give 
the  pilot  a  chance  to  practice  what  he  had  learned. 
The  simulator  was  not  training  program. 


training  program,  there  was  no  attempt  to  teach 
everything  in  it.  If  the  simulator  did  not  have  the 
capability  to  enable  training  a  particular  maneuver 
there  were  not  any  sc, ua res  to  fill  to  show  that  we 
tried. 

Third,  the  simulator  was  used  to  make  what  flying 
time  we  did  have  more  effective.  Even  in  1967,  flying 
time  was  not  in  abundance.  What  limited  training 
time  we  had  was  valuable.  We  used  the  time  in  the 
simulator  to  train  to  a  le^el  commensurate  with  the 
simulator's  capabilities.  It  was  never  considered 
that  we  would  use  simulator  time  to  replace  flying 
time.  We  needed  the  simulator  time  to  make  our  flying 
time  more  meaningful,  productive,  and  safe. 

Fourth,  _r.d  this  goes  along  with  tv>  and  thr#»p. 
we  didn't  worry'  about  "fidelity"  as  an  end.  The 
important  tiling  was  the  ability  to  train  a  particular 
maneuver  effectively.  Sure,  it  handled  poorly,  but 
it  was  understood  that,  if  we  could  fly  the  simulator 
and  handle  emergency  and  instrument  procedures,  the 
real  thing  would  be  a  breeze.  From  experience  I  can 
tell  you  that  this  was  true. 

Fifth,  and  probably  most  important  was  the 
ability  and  attitude  of  the  instructors.  Teaching  in 
the  simulator  was  not  rotated  among  whomever  could  be 
conned  into  teaching  in  the  simulator.  The  simulator 
instructor  position  was  a  truly  selective  position 
and  individuals  who  were  assigned  to  the  position  were 
the  ones  who  had  the  inside  track  for  good  report  cards 
and  promotions.  As  a  result,  we  rarely  lacked 
quality  instruction.  The  instructors  were  profes¬ 
sional,  kncwledgable  and  excellent  teachers.  This 
made  a  real  difference.  As  a  testimony  to  the 
quality  of  this  program,  the  simulator  was  giver  much 
of  the  credit  for  over  500,000  accident-free  flying 
hours  that  the  375th  Aeromedical  Airlift  Wing  enjoyed 
operating  20-year  old  aircraft.  When  the  aircraft 
were  retired  in  the  early  seventies,  their  accident- 
free  record  renamed  untarnished. 

So  what's  the  point?  What  can  we  learn  from  this 
experience?  For  years  we've  been  meeting  like  this 
and  telling  each  othur  what  wonderful  things  we're 
doing,  have  done,  and  hope  to  do  in  the  future  for  the 
simulator  world.  In  the  process,  we've  looked  at  the 
pilot  "through  a  microscope."  We've  analyzed  him, 
scrutinized  him,  and  studied  him.  We've  studied  his 
aircraft,  his  mission  and  his  bodily  functions. 
We've  made  great  strides  in  technology,  digitized 
computers,  expanded  fields  of  view,  increased 
resolution  and  focussed  on  fidelity.  Our  reward  for 
all  our  work  has  been  a  pilot  who  would  much  rather 
play  "pac-man"  than  train  in  our  $10-$100  million 
electronic  training  devices.  Have  we  done  something 
wrong?  If  so,  where  have  we  gone  wrong?  How  can  we 
do  it  better?  Certainly  with  the  quality  of  our 
equipment  our  programs  should  be  able  to  easily  exceed 
the  effectiveness  of  earlier  prograns  like  Air 
Evac's.  To  me  the  problems  are  observable, 
predictable,  and  correctable. . .but  not  easily. 

These  problems  are  tied  up  in  four  words: 

1.  Overestimate 

2.  Underanticipate 

3.  Overemphasize 

4.  Overcomplicate 

There  isn't  any  one  sector  that  can  be  identified  as 
the  guilty  party.  Everyone,  yes  everyone,  has  had  a 
part.  These  include  developers,  the  contractors, 
acquisition  agencies,  the  Pentagon,  command 
headquarters,  testing  agencies,  requirements  people, 
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and. . .yes. . .the  pilots  themselves. 

The  first  word  is  overestimate,  specifically  the 
simulator's  capability  to  meet  all  our  needs  in  a 
certain  limited  time.  To  me  this  tendency  started 
with  the  oil  embargo  in  1973.  Ever  noticed  how 
fashionable  it  is  these  days  to  trace  all  our  problems 
back  to  the  Arab  oil  embargo.  Anyway,  up  until  then, 
simulators  had  been  going  along  fine  in  their  proper 
role  and  gaining  in  capability.  Then  someone  got  the 
bright  idea  that  we  could  use  simulatois  to 
replace  flying  time  rather  than  just  increasing  the 
effectiveness  cf  it.  Therefore  the  more  we  used 
simulators  the  less  we  needed  airplanes  and  the  more 
we  could  save  gas.  The  conclusion  was  that 
simulators  were  pretty  good  but  many  dreamed  that  with 
seme  good  old  American  ingenuity  simulators  could  be 
developed  so  that  pilots  would  never  have  to  leave  the 
ground. . .except  in  an  emergency,  of  course. 

This  led  to  a  flurry  of  technological  efforts  and 
flight  hou.t  tradeoff  studies.  Some  elements  of  the 
Air  Force  committed  thanseives  to  giving  up  flight 
hours  in  exchange  for  a  certain  simulator  capability. 
These  estimates  were  based  on  projections  of  the 
expected  technological  advances.  Unfortunately  the 
technology  was  not  all  that  was  expected  as  soon  as 
expected.  Furthermore  the  estimates  had  been  fudged 
a  little  to  make  the  case  look  better  because  surely 
they  wouldn't  take  avay  all  that  flying  time. 

Now  consider  the  pilot.  He  knew  instinctively 
that  he  could  not  minimize  his  flight  time  to  the 
extent  that  his  proficiency  deteriorated.  But 
everyone  said  "Trust  me,  you'll  really  be  impressed 
with  its  capability.  Reluctantly,  he  said  "OK". 

Five  years  later  the  simulator  arrives.  It's 
two  years  late  and  the  flying  hour  cuts  have  taken 
place  two  years  earlier.  Unfortunately  the  costs  of 
the  full  visual  capability  have  escalated  and  it's 
been  cut  from  the  program.  Besides  there  wasn't 
enough  spare  memory  in  the  computers  to  handle  the 
visual  system.  Also  the  flying  qualities  aren't  the 
same  as  the  airplane  because  the  simulator  was  built 
on  design  data,  and  flight  data  wasn't  available  until 
after  the  critical  design  review.  That  was  four 
years  ago  but  the  program  couldn't  afford  the  oost 
growth  that  would  be  required  to  use  flight  test  data. 
Or  we  have  the  flight  test  data  but  it  doesn't  have 
enough  data  points  or  our  sampling  rate  needs  to  be 
larger  or  any  nuni>er  of  a  myi.iad  of  technical  reasons 
why  it  isn't  quite  right  or  wasn't  delivered  on  time. 
The  base  newspaper  has  an  article  about  the  simulator 
and  says  that  it  has  just  passed  its  reliability 
testing  with  flying  colors.,  However,  one-half  hour 
into  the  first  mission  it  has  five  computer  halts 
These  are  explained  as  merely  software  "gliches"  that 
were  not  reflected  in  the  reliability  data  because 
software  does  not  fail.  I  could  go  on  and  on,  but  why 
beat  a  dead  horse.  Simply  classify  '  t  under  the  first 
word:  We  overestinuted  what  we  could  do. 

The  second  word  is  unde: anticipate.  It  could 
also  be  three  words:  lack  vi  olanninu.  It  seons  in 
the  last  few  years  many  simulators  arrive  on  base  just 
about  the  same  time  as  the  training  syllabus.  One  of 
the  neat  things  about  the  Air  EVac  simulator  was  that 
it  was  part  of  the  ground  training  program.  It's 
obvious  that  the  training  program  should  be 
conceptualized  and  then  it  should  be  determined  where 
a  simulator  or  other  training  device  could  be  used  to 
effectively  train  what  has  to  be  trained.  Trainer 
features  seem  to  be  obtained  like  someone  lr.  the 


grocery  store  without  a  list:  whatever  sounds  good 
is  what  we  order.  There  seems  to  be  minimal  thought 
placed  in  (1)  developing  a  syllabus,  and  (2) 
requesting  those  features  which  will  best  fit  in  with 
that  syllabus. 

Overemphsize  is  the  third  word  and  fidelity  is 
the  one  that  goes  along  with  it.  Simulation  is 
exactly  that:  simulation.  By  definition,  no 
simulator  will  ever  have  total  fidelity. 
Furthermore-  total  fidelity  doesn't  guarantee  an 
excellent  training  device.  The  real  airplane  has 
complete  fidelity,  but  is  only  an  excellent  training 
device  with  a  competent  instructor.  The  Air  Evac 
simulator  had  very  little  "fidelity",  but  the  way  it 
was  used  made  it  an  extremely  capable  training  device 
with  a  great  deal  or  training  capability.  Our 
preoccupation  with  "fidelity"  has  driven  up  the  cost 
and  complexity  of  simulators  and  detracted  from  their 
training  capability. 

Consider  the-  pilot  in  the  field  again.  He  hears 
everyone  talking  about  fidelity.  Therefore,  when  he 
goes  to  the  simulator  he's  looking  for  something  to  be 
not  quite  like  the  aircraft.  He  doesn't  have  to  look 
far,  and,  no  matter  how  much  money  we  spend,  he  will 
always  be  able  to  find  things  that  are  not  quite  like 
the  aircraft.  And,  if  all  we're  concerned  about  is 
fidelity,  we  can  certainly  get  that  much  cheaper  and 
quicker  in  the  aircraft.  Rather  we  should  talk 
training  capability.  That's  what  we're  after,  isn't 
it?  And,  it  should  be  training  capability  for  those 
portions  o  ±  the  mission  for  which  the  simulator  is 
best  suited  and  designed. 

The  last  word  is  overcomplicate,  and  it's  really 
a  synthesis  of  the  other  three.  Our  preoccupation 
with  fidelity  leads  to  expensive  systems  that  are 
difficult  to  maintain  and  costly  to  operate. 
Furthermore  we  deemphasize  training  capability.  Our 
failure  to  design  the  training  program  in  advance  of 
the  simulator  leads  to  the  "give  me  everything  there 
is  and  we're  bound  to  build  an  effective  training 
program"  approach.  The  problem  is  that  it's  always 
too  expensive,  sore  thing  always  gets  cut,  and  there  is 
no  way  to  evaluate  what  is  most  irqportant  because 
there  isn't  any  training  progi  am  with  which  to  compare 
it.  A  good  example  is  that  many  of  the  instructor 
stations  have  many  expensive,  underutilized  systems. 
Had  there  been  a  development  of  the  training  program 
first  these  could  have  been  identified  as  unnecessary 
and  the  funds  diverted  to  other  more  valuable 
features. 

Well,  those  are  the  problem.;  as  I  see  them.  And 
it  has  been  said  many  times  that  anyone  can  be  a 
critic.  What  do  you  do  to  make  it  better?  The  f  irst 
reccmmendation  is,  of  course,  to  design  the  training 
program  first .  The  simulator  should  then  be 
appropriately  integrated  into  the  training  program. 
This  is  rot  new  advice,  but  it  still  holds  true.  With 
this  kind  of  approach  you  can  take  a  look  at  the  tasks 
you  need  to  train  most  and  let  the  developer  work  to 
obtain  that  capability.  This  also  precludes  the 
tendency  to  ask  tor  the  moon. 

Secondly,  be  realistic  with  schedules.  The 
acquisition  agency'  puts  out  a  request  for  delivery  of 
a  simulator  in  three  years,  knowing  this  is  an 
unattainable  goal.  The  ratiorale  for  this  procedure 
is  based  on  deliveries  bc-ing  historically  late.  The 
contractor,  to  ensure  contract  award  concurs  with 
this  schedule. 
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Don't  forget  the  software,  documentation  and 
spare  parts.  As  a  pilot  I  must  admit  these  are 
three  ruzzy-wuzzy  areas  in  my  mind  except  when  the 
simulator  doesn't  work,  breaks  down  and  can't  be 
fixed.  Mumbo-jumbo  about  the  reliability  figures 
do  not  consider  software  failures  as  failures,  the 
level  of  documentation  that  we  ordered  doesn't 
cover  this,  and  "this  part  has  'downed'  the  simu¬ 
lator  but  we  won't  be  able  to  get  it  for  six 
months"  doesn't  mild  confidence  in  the  capability 
of  these  devices. 

The  last  one  is  probably  the  toughest  one  and 
the  key  tn  whole  program:  obtaining  good, 
motivated,  exceptional  instructors.  This  involves 
changing  an  entire  attitude  about  simulators. 
In  many  cases,  simulator  programs  have  become  the 
dumping  ground  for  passovers.  And  if  they're 
not  passovers  when  they  get  there,  they  are  soon 
after  because  they  are  not  recognized  for  their 
contr ibut ions . 

Simulators  have  considerable  capabilities. 
They  can  enhaiice  and  improve  any  training  program. 
The  important  thing  to  remember  is  they  cannot 
replace  an  airplane,  or  enhance  a  poorly  conceived 
and  executed  training  program.  Thank  you. 
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ABSTRACT 


The  purpose  of  this  paper  is  to  delineate  the  various  responsibilities  and  inter¬ 
relationships  of  the  agencies  involved  in  the  acquisition  of  a  major  training  system. 
-As  th£ ■'’rt+.le.  suqoests^  ,£his  paper  will  emphasize  the  role  of  the  user  as  perceived  by 
the  materiel  developer.-^  Discussed  within  the  will  be  these  interrelationships 

as  they  exist  with  a  major  Army  training  system  acquisition.  The  participants  will  oe 
identified,  the  four  phases  of  the  acquisition  process  will  be  presented  and  a  dis¬ 
cussion  of  the  role  of  the  user  in  this  process  will  be  highlighted.  Also  covered 
will  be  the  user's  role  in  the  development  of  Front-End  Analysis  (FEA)  materiels  and 
Task  and  Skills  Analysis  (TASA)  documentation.  In  addition,  a  common  vocabulary  will 
be  included  to  establish  a  basis  of  understanding  and  suggestions  will  be  offered  on 


problems  which  need  heightened  interest  and 
and  acquisition  processes.^ 

INTRODUCTION 

| 

In  Army  training  system  development,  there  are 
primarily  four  groups  or  agents  who  play  a  crit¬ 
ical  role.  These  include  the  materiel  developer, 
the  training  developer,  the  user  school  (or  pro¬ 
ponent)  and  the  selected  contractor  who  will  pro¬ 
duce  the  training  system  or  device.  Although 
there  are  numerous  other  agencies  and  participants 
in  the  entire  training  system  acquisition  cycle, 
for  the  needs  of  this  paper,  only  these  four  crit¬ 
ical  participants  will  be  addressed.  Once  identi¬ 
fied,  the  role  of  these  agencies  will  be  matched 
to  the  acquisition  process  itself.  Focusing  on 
the  user  proponent  and  its  role  in  the  process, 
suggestions  will  be  offered  as  to  how  a  more  pro¬ 
ductive  re’ationship  between  agencies  may  be 
obtained.  The  user's  role  in  the  development  of 
FEA  end  TASA  materiels  will  also  be  addressed. 

ACQUISITION  CYCLE  PARTICIPANTS 

A  description  of  the  four  participants  iden¬ 
tified  as  critical  for  this  discussion  can  be 
found  in  the  PM-TRADE  Training  Device  Acquisition 
and  Life  Cycle  Management  Guide  as  prepared  <:ur 
the  Project  Manager  Training  Devices,  U.S.  Anny.(') 
For  the  purposes  of  this  paper,  these  participants 
are  the  Training  Developer,  Materiel  Developer, 
Proponent  School  (User)  and  selected  contractor 
who  will  develop  the  training  system  or  device. 

The  Training  Developer  is  that  agency  respon¬ 
sible  for  the  formulation  of  training  concepts, 
doctrine,  organization,  training  objectives  and 
requirements  for  the  training  of  U.S.  Army  "orces. 
This  command  or  agency  is  responsible  for  tne 
development  and  conduct  of  training  which  will 
provide  the  skills  necessary  to  operate  and 
logistically  support  materiel  systems  being 
developed.  In  most  instances,  *‘ie  principal 
Training  Developer  is  the  U.  S.  Army  Training 
and  Doctrine  Command  (TRAD0C). 

The  Materiel  Developer  is  the  element  respon¬ 
sible  for  research,  development,  production  and 
production  validation  of  a  training  d?vice.  In 


tracking  during  the  design,  development 


most  cases,  the  Project  Manager  for  Training 
Devices  is  the  principal  Materiel  Developer.  The 
Project  Manager  (PM)  of  the  systems  involved  (e.g., 
M-l  Tank,  U.S.  Roland,  Firefinder,  AH-54)  is 
generally  charged  to  oversee  .he  total  project  and 
provide  funding.  Working  with  and  within  this 
agency  is  the  Project  Director  at  PM-TRADE  who  is 
assisted  by  the  Naval  Training  Equipment  Center. 

The  responsibility  here  is  to  turn  a  proponent 
school's  (user's)  training  requirements  into 
specifications  for  inclusion  in  a  Request  for 
Proposal  (RFP).  This  is  the  document  that  provides 
a  description  of  the  items  to  be  procured.  It  is 
a  request  for  a  manufacturer  to  submit  a  proposal 
supported  by  a  cost  breakdown.  By  directing  this 
effort,  the  Project  Director  is  considered  the 
driving  force  for  PM-TRADE  in  the  accomp1 ishment 
of  the  mission  of  training  device  acquisition. 

The  User  (or  proponent  school)  is  designated 
as  the  command,  organization,  or  unit  that  ij  to 
receive  the  training  system  or  device  from  pro¬ 
duction  for  use  in  accomplishing  a  designated 
mission.  The  user  provides  guidance  to  the 
materiel  and  training  developers  during  the  train¬ 
ing  device  acquisition  process  on  matters  pertain¬ 
ing  to  the  expected  operational  employment  and 
logistic  support.  Although  not  always,  the  user 
or  proponent  is  normally  the  agency  who  supplies 
to  HQ  TRADOC  a  training  device  requirement  for 
validation.  This  validation  process  encompasses 
the  establishment  of  a  Joint  Working  Group  con¬ 
sisting  of  representatives  of  PM-TRADE,  USATSC 
(U.S.  Army  Training  Support  Center  -  the  ♦raining 
device  focal  point  for  TRADOC),  and  the  Proponent 
School  (Armor,  Infantry,  Air  Defense,  Artillery, 
etc.)  with  the  DARCOM-PM  (Dept.  Army  Readiness 
Command-Project  Manager).  It  is  the  Training 
Device  Requirement  (TDR)  document  prepared  by 
TRADOC  which  gives  operational,  technical,  and 
cost  information  necessary  to  obtain  Headquarters . 
Department  of  the  Army  (HQDA)  approval.  When 
approved  by  HQDA,  the  TDR  is  the  document  of 
record  of  the  Amy's  requirement  and  will  contain 
the  guiding  factors  against  which  developers  and 
contractors  meet  the  user's  needs. 
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i  ne  seieceeu  mn  n  uv.  ivi  ij  . . ^  , 

partnership,  company,  corporation  or  association 
having  a  contract  with  the  procuring  activity 
(usually  PM-TRADE  with  the  contracting  auspices 
of  Naval  Training  Equipment  Center)  for  the  design, 
development,  manufacture,  maintenance  modifica¬ 
tion,  or  supply  of  items  under  the  terms  of  a 
contract.  In  the  cate  of  a  training  device,  this 
participant  builds  th?  hardware  and  develops  the 
training  scenarios  anc  software  as  per  the  specifi¬ 
cation  requirements  of  the  contract.  This  is  the 
legal  agreement  between  the  Government  and  Industry 
for  the  acquisition  of  the  device  needed. 

Within  this  paper,  these  four  participants 
will  be  of  concern  with  the  user's  role  high¬ 
lighted,  in  particular  with  respect  to  the  train¬ 
ing  device  acquisition  process. 

TRAINING  DEVICE  ACQUISITION  PROCESS 

The  training  device  acquisition  process  con¬ 
sists  of  four  distinct  phases.  They  include  (1) 
Need  Identification  -  Concept  Formulation  Phase, 

(2)  Demonstration  and  Validation  Phase,  (3)  Full- 
Scale  Engineering  Development  Phase,  and  (4  Pro¬ 
duction  and  Deployment  Phase. 

The  Need  Identification  -  Concept  Formulation 
Phase  is  where  training  voids,  new  training  needs, 
and  technological  forecasts  are  identified  by  the 
Training  Developer  to  determine  the  training  capa¬ 
bilities,  doctrine,  organization,  or  potential 
training  devices  that  will  improve  the  training 
of  the  Army.  The  Demonstration  and  Validation 
Phase  is  where  technical  concepts  are  validated  to 
determine  if  they  fulfill  the  needs  or  voids  that 
were  identified  and  that  training  effectiveness  is 
achievable.  In  the  third  phase,  FuVh-Scale 
Engineering  Development  Phase,  the  training  device 
or  system  is  fully  developed,  engineered,  fabri¬ 
cated,  tested  and  a  decision  is  made  whether  the 
item  or  system,  is  acceptable  to  meet  the  require¬ 
ment.  In  the  fourtn  and  final  phase,  Production 
and  Deployment  Phase,  the  training  system  or 
device  is  procured  and  distributed,  individuals 
or  groups  are  trained  in  its  use,  and  logistic 
support  is  provided.  Within  these  phases, 
normally  the  procurement  and  contract  administra¬ 
tive  services  are  provided  by,  or  through,  the 
Contracting  Officer  from  the  Naval  Training 
Equipment  Center,  Orlando,  Florida. 

LINES  OF  RESPONSIBILITY 

After  examining  the  participants  and  the 
process  of  training  device  acquisition,  it  would 
be  helpful  to  discuss  the  working  relationships 
involved.  Establishing  clear  lines  of  respon¬ 
sibility  and  coordination,  plus  review  and 
guidance  channels  among  the  participants  in  the 
acquisition  process  is  difficult  at  best.  The 
syi tern  for  achieving  this  coordination  is 
theoretically  in  place,  but  more  often  than  not 
during  the  acquisitior  *  a  major  training  system 
or  device,  mar./  of  the  participants  at  one  time 
or  another  are  "overtaken  by  events"  and  these 
channels  ine-itably  break  down.  This  breakdown 
can  be  caused  by  a  number  of  reasons  -  it  could 
be  simply  a  lack  of  communication  and'or  the 
inability  to  establish  clearly  defined ‘1 ines  of 


ticipants  in  the  acquisition  process  is  one  umi 
requires  dedicated  coordination  and  clearly  defined 
areas  of  responsibility  to  ensure  timely  and 
accurate  input  into  training  requirements  and  the 
resultant  training  system  or  device  development 
needed  to  fulfill  these  needs.  A  breakdown  between 
developer  and  user;  between  developer  and  con¬ 
tractor  or  any  combination  of  participants,  could 
result  in  an  inadequate  or  inappropriate  develop¬ 
ment  procuct. 

THE  USER'S  ROLE 

The  user's  (or  proponent  school's)  involvement 
in  the  acquisition  process  should  be  one  of  active 
participation  from  the  early  stages  of  requirements 
definition  and  TDR  development  up  through  and 
beyond  the  actual  design,  development,  acquisition 
and  delivery  of  the  training  system  or  device. 

The  user's  role  begins  with  the  initial  identifica¬ 
tion  of  a  training  need.  At  this  point  a  complete 
FEA  should  be  conducted  by  the  proponent  to 
determine  how  the  training  system  or  device  would 
fit  into  the  total  training  program.  This  analysis 
addresses  the  how,  who,  when,  why  and  where  aspects 
of  the  proposed  utilization  of  the  training  system 
or  device.^  Because  the  Army  is  committed  to 
the  "Systems  Approach"  for  the  development  and 
acquisition  of  major  materiel  it-,  ms, (4)  it  is 
mandatory  that  the  user  be  versed  in  the  Instruc¬ 
tional  Systems  Development  (ISD)  process  and  be 
able  to  work  confortably  within  its  guidelines. 
Front-end  analysis  as  part  of  the  ISD  process  has 
beer  described  as  the  "imeractive  process  by 
means  of  which  the  requirements  of  a  system  may  be 
progressively  more  definitive  and  brought  more 
sharply  into  focus. This  implies  that  FEA  is 
an  on-going  and  dynamic  process  that  requires  the 
authors  to  up-date  and  "fine  tune"  the  information 
presented  as  the  knowledge  base  about  training 
requirements  and  reeds  expand.  In  most  cases, 
part  of  this  requirement  a  icilysis  will  include 
Task  and  Skill:,  \f-alvsis  x  CASA)  of  a  present  or 
nearly  deployed  weapon  system  for  use  in  the 
development  of  a  inquired  training  device.  This 
Task  Analysis  will  not  only  serve  as  a  basis  for 
the  Training  Device  Requirement  (TDR)  but  could 
conceiveably  be  part  of  a  government  deliverable 
to  the  contractor  to  form  a  baseline  from  which 
to  work.  It  is  because  of  the  fact  that  the 
acquisition  process  embraces  all  phases  from 
inception  of  a  training  need  to  completion  of  a 
contract^)  that  the  quality  of  the  front-end 
input  is  so  important  in  the  success  of  develop¬ 
ing  a  needs  responsive  device  or  training  system. 
Central  to  this  idea  of  quality  input,  is  the 
associated  need  of  continued  input  and  communica¬ 
tion  between  user  and  materiel  developer  through¬ 
out  the  acquisition  cycle.  This  is  most  critical 
when  the  materiel  developer  is  monitoring  and 
reviewing  the  work  of  the  contractor.  As  an 
example,  after  contract  award  and  during  actual 
development  of  a  training  device,  problems  some¬ 
times  arise  concerning  the  exact  definition  or 
training  intent  for  the  device  that  is  to  be 
procured,  or  possibly  a  change  in  training  needs 
na$  developed.  The  quality  of  communication  and 
input  that  takes  place  between  the  materiel 
developer  and  user  can  determine  the  effectiveness 
and  useability  for  the  training  device  when 
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delivered.  A  lack  of  quality  input  or  a  lack  of 
timely  input  can  both  result  in  inappropriate 
training  design  and  can  slow  the  development  pro¬ 
cess,  possibly  causing  additional  ex  nse.  As 
mentioned  before,  clarification  of  u^er  need  is 
dynamic  and  always  improving  in  terms  of  pre¬ 
cision  and  accuracy  of  training  requirements. 
Certainly,  the  user  can  be  instrumental  in  help¬ 
ing  the  materiel  developer  make  critical  deci¬ 
sions  on  such  items  as;  which  task  to  train, 
review  of  instructor  display  formats  on  training 
devices  and  the  requirements  for  student  perfor¬ 
mance  records,  to  name  just  a  few.O)  The 
addition  of  this  requirement  to  the  school's  list 
of  responsibilities  -  platform  instruction,  train¬ 
ing  extension  courses,  field  manual  development, 
and  training  device  requirements  development, 
etc. (3)  is  indeed  a  tall  order,  but  one  that  will 
pay  dividends  in  the  form  of  increased  training 
effectiveness  of  the  devices  being  developed.  The 
better  the  quality  and  timeliness  of  the  school's 
input  to  the  materiel  developer  during  this  stage 
of  the  acquisition  process,  the  better  the  quality 
of  the  device  itself. 

SUGGESTIONS  FOR  IMPROVEMENT 

If  the  user  schools  are  to  provide  the  needed 
input  to  the  acquisition  cycle  in  both  a  quality 
and  timely  fashion,  it  is  necessary  that  they  be 
supplied  with  the  needed  resources  and  personnel 
to  do  the  job  correctly.  It  is  important  that 
the  members  of  the  acquisition  team  do  not  have 
their  abilities  and  energies  stretched  too  thin. 
Subject  matter  experts,  tacticians,  instructors 
and  instructional  system  developers  are  needed 
to  supply  the  required  information.  Those  indivi¬ 
duals  assigned  the  responsibility  for  these,  areas 
must  also  use  their  role  as  decision-makers. 

Their  opinions  and  direction  are  required  to  help 
improve  this  training  device  acquisition  cycle. 

When  these  qualities  are  lacking,  the  materiel 
developer  and  others  involved  in  the  acquisition 
process  are  left  to  their  own  devices  to  secure 
the  appropriate  answers.  A  deferred  user  decision 
is  a  decision  someone  else  will  make.  The  user 
will  be  ultimately  required  to  live  with  this 
decision. 

A  second  improvement  could  be  made  if  a  better- 
system  of  "corporate  memory"  were  instituted  within 
the  schools.  This  might  be  achieved  by  maintain¬ 
ing  a  more  stable  caore  of  individuals  who  can 
stay  with  a  project  from  inception  to  completion. 
This  would  help  solve  the  "learning  curve  problem" 
and  "memory  loss"  associated  with  a  high  turnover 
of  personnel.  Working  with  the  materiel  develope* 
and  all  members  of  the  acquisition  process,  the 
user  must  keep  abreast  of  program  direction  and 
maintain  a  steady  flow  of  input.  This  will  assure 
that  the  school's  views  and  ne.ads  are  kept  in 
the  forefront.  To  do  this  effectively,  it  is 
mandatory  that  the  "corporate  memory"  of  the 
school  be  maintained  and  when  personnel  change* 
are  required,  a  consented  effort  is  made  to  bring 
the  oe-w  member  up  to  speed,  and  to  enable  him  to 
Decome  a  productive,  contributi ve  segment  of 
the  team. 

A  final  suggestion  would  be  the  improvement 
of  the  communications  lin\  be* ween  the  matt-riel 


developer  and  the  user  j>chool.  All  too  often 
vital  information  needed  for  steady  pro¬ 
gression  of  the  acquisition  cycle  "finds  the 
cracks  and  falls  through"  for  lack  of'  an  efficient 
communications  link.  Establishing  c /  more  efficient 
communications  network  between  the  /level oper  and 
user  could  go  a  long  way  in  improving  the  recep¬ 
tion  and  use  of  the  delivered  materiels.  It  must 
be  stressed  that  the  responsibility  of  seeing  that 
tnis  communication  link  exists  is  as  much  the 
school's  responsibility  as  it  is  the  materiel 
developer's.  In  fact,  it  is  in  the  school's  or 
user's  best  interest  to  keep  t h i */,  link  connected 
and  used  to  its  fullest  benefit.  When  the  user 
keeps  the  developer  informed  and  up-to-date  as  to 
what  the  user's  needs  are,  everyone  benefits. 

CONCLUSIONS 

The  user's  role  in  major  training  system 
acquisition  from  the  perspective  of  the  materiel 
developer  is  one  of  active  participation  from  the 
early  stages  of  requirements  definition  to 
delivery  of  the  training  system.  Quality  and 
timely  input  at  ail  required  stages  of  the  process 
will  pay  returns  in  the  form  of  a  training  effec¬ 
tive  product.  Tn  this  respect,  it  is  vitally 
important  that  the  user  assume  a  leadership  role  in 
providing  the  necessary  information  concerning  FEA 
and  TASA  materiels  needed  to  develop  an  effective 
system,  and  continue  this  role  consistently 
throughout  the  entire  process  of  training  needs 
identification,  design,  development  and  acquisi¬ 
tion  of  the  system  involved.  Given  the  expertise, 
dedication  and  ability  of  the  personnel  at  the 
user  schools,  it  is  imperative  that  an  effective 
communications  link  be  established  between 
developer  and  school  to  tap  this  invaluable 
resource  and  use  it  to  its  best  advantage  during 
the  training  system  acquisition  process. 

GLOSSARY 

The  list  included  here  is  intended  to 
help  establish  a  common  base  of  defini¬ 
tion  of  some  of  the  terms  used  through¬ 
out  the  acquisition  cycle.  Although 
not  all  inclusive,  this  list  includes 
many  of  the  terms  used  in  this  paper. 

Acquisition  Life  Cycle.  Normally  consists  of  four 
tses  (Concepts,  Validation,  Full-Scale  Engineer¬ 
ing  Development,  and  the  Production  and  Deployment) 
with  key  decision  points  reacned  at  program  initia¬ 
tion  and  between  each  of  the  phases  for  major 
systems.  These  phases  cxnlain  a  normal  acquisition 
path,  not  a  prescribed  path,  which  ail  programs 
must  follow,  A  program  may  skip  a  phase,  have 
program  elements  in  any  or  all  phases,  or  have 
multiple  decision  points  per  phase 

Materiel  Developer  (MD).  The  command  or  agency 
responsible  for  research,  development  and  produc¬ 
tive  validation  of  a  system  (to  include  the  system 
for  its  logistic  support)  which  responds  to  HO  DA 
objectives  and  requi rements.  Materiel  developers 
designated  from  the  following,  with  specific 
responsibilities  assigned  as  appropriate:  Chief 
of  Engineers,  The  Surgeon  General;  CG  DARCOM 
CG  U SAC SC :  CG  USAINSCOM:  and  CO.  U.S.  Army 
Research  Institute  ( AR I ) . 
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Product  Manager  (PM).  The  individual,  designated 
by  a  Materiel  Developer,  who  is  delegated 
authority  and  assigned  responsibility  for  central¬ 
ized  management  of  a  particular  development/ 
acquisition  or  other  specified  program  that  does 
not  qualify  for  system/program  project  management 
but  requires  some  degree  of  centralized 
management. 

Project  Manager  (PM).  Ar.  ind;/idual,  chartered 
by  the  CG.  DARCOM,  who  is  assigned  the  responsi¬ 
bility  and  is  delegated  the  full -line  authority 
for  the  centralized  management  of  a  specified 
development/acquisition  project. 

Proponent  School.  The  TRADOC  school  designated 
by  the  CG,  TRADOC,  to  exercise  supervisory  manage¬ 
ment  of  all  combat/training  development  aspects  of 
a  materiel  system. 

Request  for  Proposal  (RFP).  Request  for  the 
manufacturer  to  submit  a  proposal  supported  by 
a  cost  breakdown.  It  provides  a  description  of 
the  items  to  be  procured.  It  may  include 
specifications,  Quantities,  time  and  place  of 
delivery,  method  of  shipment,  packaging  and 
instruction  manual  requirements,  materiel  to  be 
furnished,  and  data  requirements,  both  logistic 
and  administrative. 

Training  Developer.  The  command  or  agency 
responsible  fer  the  development  and  conduct  of 
the  training  which  will  provide  the  skills 
necessary  to  operate  and  logistically  support 
materiel  systems  being  developed  or  otherwise 
acquired.  (For  most  equipment,  this  is  TRADOC). 

Training  Device.  Any  three-dimensional  object 
level  oped,  fabricated  or  procured  specifically 
for  improving  the  learning  process.  Training 
devices  may  be  either  system  devices  or  non- 
system  devices.  Items  which  simulate  or  demon¬ 
strate  the  function  of  equipment  or  system  such 
as  three-dimensional  models,  mockups,  or  exhibits, 
and  are  designed,  developed,  and  procured  solely 
to  meet  training  support  requirements.  They  are 
further  defined  as  follows: 

(a)  System  devices  are  designed  for  use  with 
one  system  or  item  of  equipment,  includ¬ 
ing  subassemblies  and  components;  e.g., 
launch  effects  simulator  ^or  the  TOW 
missile  system  and  Shi  1 1  el agh  Conduct  of 
Fire  Trainer  (COFT)  for  the  MS51 ,  etc. 

(b)  Non-system  devices  are  designed  to 
support  general  military  training  and/or 
for  use  with  more  than  one  system  or  item 
cf  equipment,  including  subassemblies  and 
components;  e.g..  Multiple  Integrated 
Laser  Engagement  System,  etc. 

Framing  Device.  Acquisition  Process.  A  sequence 
of  specifier  decision  events  and  phases  of 
activity  directed  to  achievement  of  established 
program  objectives  r.  the  acquisition  of  training 
devices  and  extending  from  approval  of  a  training 
need  through  successful  deployment  of  the  system 
or  termination  of  the  program. 

Training  Device  Requirement  (TDR).  A  document 
prepared  oy  TRADOC  with  the  assistance  of 


PM-TRADE  which  gives  operational,  technical,  and 
cost  information  necessary  to  obtain  HQ  PA 
approval.  When  approved  by  HQ  DA,  the  TDR  is  the 
document  of  record  of  the  Army's  requirement  and 
will  contain  the  guiding  factors  against  which 
developers  and  contractors  meet  the  user's  needs. 

A  HQ  DA-approved  TDR  supports  the  expenditure  of 
RDTE  and/or  OPA  funds  for  development  and/or  pro¬ 
curement  of  the  training  device. 

User.  The  command,  orgar.^tion,  or  unit  desig¬ 
nated  to  receive  the  training  device  from  produc¬ 
tion  for  use  in  accomplishing  a  designated  mission. 
The  system  is  included  in  the  user's  TOE,  TDA,  or 
in  an  appropriate  Common  Table  of  Allowances  (CTA). 
The  user  provides  guidance  to  materiel  ana  train¬ 
ing  developers  during  the  training  device  acquisi¬ 
tion  process  cm  matters  pertaining  to  the  expected 
operational  employment  and  logistic  support. 

NOTE:  These  and  other  definitions  which  are  use¬ 
ful  in  understanding  the  acquisition  process  and 
the  role  each  participating  agency  plays  in  its 
completion,  can  be  found  in  the  Training  Device 
Acquisition  Management  Model  (TDAMM),  prepared 
for  the  Project  Manager  Training  Devices,  United 
States  Army. 
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ABSTRACT 
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"'This  paper  discusses  sane  af  the  problems  experienced  by  operational  users  in  their 
efforts  to  obtain  maxirrum  training  benefit  f ran  Advanced  Flight  Simulators.  The  major 
shortcomings  af  simulation  training  identified  here  are  rvf  the  result  cf  state-of-the- 
art  technology  limitatiais.  In  fact,  many  cf  the  training  difficulties  encountered  toy 
simulator  users  today  may  have  been  avoided  if  greater  attention  had  been  given  to  defining 
Mission  and  Task  Requirements  during  simulator  design.  This  suggested  enphasis  an  Mission 
indicates  a  change  cf  mind-set  is  needed  that  focuses  on  explicit  operational  requirements 
as  well  as  the  more  conventional  technical  approach.  After  analysis  cf  typical  simulator 
training  problems,  it  is  concluded  that  simulation  training  could  benefit  substantially 
through  greater  oxnsideratii.cn  to  front-end  analysis  during  simulator  design,  arid  by  pro¬ 
viding  training  personnel  with  well  designed  instructional  programs  that  includes  an 


abjective  Performance  Measurement  System. , 


INTRODUCTION 


Training  aircrews  in  such  ccnplex  tac¬ 
tical  mission  areas  as  Ai'-  Ccrrbat  Maneuvering 
(ACM)  has  always  been  difficult  at  best. 

This  is  largely  because  cf  the  dynamics  and 
complexity  oaf  these  mission  environments: 
Multiple  interrelationship?  among  numerous 
systems  must  be  perceived  and  .  anaged,  plus 
the  need,  during  critical  phases,  to  process 
substantial  information  and  respond  to  rrani- 
fold  task  requirements. 

Until  recently,  this  "higher  carder" 
training  was  aily  possible  during  actual 
aircraft  operations,  with  the  built-in  limi¬ 
tation  cf  not  being  able  to  adequately  inter¬ 
ject  sophisticated  threat  systems  into 
specific  training  missions,  as  well  as  the 
lack  of  a  means  to  repeatedly  practice 
carp  lex  mission  tasks. 


Background  and  Discussion 


national  display  technologies,  simulators  are 
new  capable  cf  providing  for  "Higher-order" 
mission  related  training  previously  not  pos¬ 
sible. 

In  view  cf  such  advances  we  would  assume 
that  operational  training  personnel  would  be 
welcoming  these  devices  with  open  arms.  Un¬ 
fortunately,  this  is  not  the  case.  Indeed, 
many  1  customers '  of  these  advanced  training 
devices  hold  them  in  low  esteem.  But,  vfoy 
is  this  sc*?  What,  exactly  is  the  problem? 

And  what,  if  anything,  can  be  done  to  iirprove 
this  low  customer  acceptability? 

"Various  significant  problems  in  operat¬ 
ing  this  air  oarbat  simulator  have  becxxne 
manifest.  Among  these  incl ude :  inadequate 
irrplementaticn  cf  instructional  design  fea¬ 
tures;  poor  design  cf  the  operation  (fea¬ 
tures)  oaf  demo  and  debrief  equipment  and 
procedures  .  .  .  .and  aemenst ration  creation 
process  lengthy  and  difficult  to  perform. 


Simulators  lave  been  around  for  quite 
seme  time,  and  most  of  us  (military  aircrews) 
have  learned  to  fly  utilizing  a  certain 
amount  cf  synthetic  flight  training.  They 
have  until  very  recently  been  designed  pri¬ 
marily  for  instruction  in  such  areas  as: 

-  Standard  Operating  Procedures  (SOP) 

-  Emergency  and  Irregularity  Procedures 

-  Inst  orient  Flight  Procedures 

-  Weapons  System  Operational  Procedures. 

Procedures  oriented  training  was,  there¬ 
fore,  about  as  rtudh  as  we  could  expect  from 
available  simulation  design  capability. 

However,  the  near  revolution  in  oaputer 
technology  has  changed  this. 

Driven  Largely  ty  advancements  in  infor- 


The  above  statement  was  extracted  from  a 
formal  (qualitative  review  cf  a  recently  in¬ 
troduced  Full  Mission  Flight  Simulator  de¬ 
signed  to  support  Navy  fighter  training.  A 
close  lock  at  these  and  other  similar  pro¬ 
blems  reveals  'chat  (1)  they  are  endemic  to 
the  recently  introduced  class  of  advanced 
flight  similators  designed  to  instruct  in 
cxuplex  mission  areas,  and  (2)  these  problems 
are  largely  operational,  vice  technical  in 
nature. 

In  addressing  these  "  operational"  diffi¬ 
culties,  this  paper  attenpts  to  pinpoint  sig¬ 
nificant  problem  areas  and  their  underlying 
causal  factors  -hich  are  resulting  in  lev 
cust.arer  acceptability.  Additionally,  a  new 
conceptual  approach  for  viewing  advanced 
similar  icn  systems  is  suggested,  as  well  as  a 
discussion  of  the  {Kssiole  application  of 
this  conceptual  framework  in  subsequent 
desigi  and/or  modif ientiens  to  these  sophis¬ 
ticated  training  systeine. 
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It  should  be  aiphasized  that  it  is  not 
the  intention  of  this  treatment  of  simulation 
design  to  single  ait  and  criticize  any  indi¬ 
vidual  group  or  place  anyone  in  bad  light.  By 
candidly  discussing  certain  operational  dif¬ 
ficulties  and  suggesting  a  possible  course  of 
action  it  is  hoped  that  a  dialogue  can  be 
established  between  operational  users  and  the 
simulator  design  oarmunity  in  order  to  im¬ 
prove  overall  mission  performance  of  these 
inport  ant  training  systems . 

Problem  Statement 


Mission  Area 

One  vs.  One 

Tactics 

(lvsl) 


Missile 

Envelope 


Deficiency 


o  Only  two  models  used  for 
Wide  Angle  Visual  (WAVS) 
presentation.  Need  addi¬ 
tional  aircraft  models  to 
simulate  full  threat 
spectrum. 

o  Air-to-air  threat  missiles 
are  net  simulated.  Need 
reasonable  inventory  of 
threat  missile  systems. 


While  it  was  orginally  believed  that 
most  simulator  shortcomings  were  either  the 
result  of  technological  limitations  or  to  a 
lesser  extent  funding  constraints,  a  reoent 
problem  analysis  conducted  by  the  author 
revealed  that  nany  major  systems  deficiencies 
fall  substantially  oitside  these  categories. 
Our  major  difficulties,  it  seems,  do  not  stem 
from  these  areas  -  cur  technology  is  maturing 
nicely  and  considerable  f  'nding  appears  to  be 
aval  l  ,i  Le  for  adequate  design  -  but  stems 
lari  i/  from  the  absence  cf  a  coherent  "mis¬ 
sion  logic"  to  drive  the  cognizant  simulator 
design.  Moreover,  since  this  conventional 
design  process  usually  ccmnences  with  uesign 
specifications  which  remain  largely  tech¬ 
nical,  inadequate  focus  is  provided  for  many 
critical  mission  requirements.  Consequently, 
inport  ant  mission  considerations  during  the 
ccnoept.ual  design  phase  are  either  super¬ 
ficially  addressed  or  totally  ignored. 

A  comprehensive  analysis  of  simulat  ;r 
problems  brought  to  light  the  following 
fundamental  areas  where  significant  efforts 
should  be  directed. 

o  A  change  of  mind  -set  appea  rs  to  be 
needed  to  move  from  an  exclusive  tech¬ 
nical  focus  to  a  broader  view  which 
fully  embraces  mission  requirements  and 
instructional  design  requirements  as 
well. 

o  A  mechanism  is  needed  to  spotlight  cri¬ 
tical  mission  areas,  thereby  bringing  to 
bear  appropriate  resources  so  that 
enhanced  training  <an  be  conducted  to 
inpreve  performance  in  these  critical 
regions. 

Although  nany  of  air  advanced  full  mis¬ 
sion  simulators  are  experiencing  certain 
technical  difficulties,  mast  cf  these  are 
either  being  corrected  or  corrective  action 
is  being  contemplated.  Q*  the  ether  hand, 
tine  difficulty  of  many  of  these  same  devices 
to  adequately  instruct  in  important  mission 
areas  for  vhich  they  are  designed,  poses  a 
more  serious  prdb’jtt  leavise  it  striker  at 
tine  very  lieart  /.  tiv_  conventional  inesiejn 
process.  For  example,  a  recently  installed 
full  mission  flight  simulator  for  West  Ccnft 
Navy  filter  training  exhibits  the  following 
“mission  related"  deficiencies  nrfnicin,  it 
appears,  should  have  been  addressed  during 
tine  original  c*u£iqn  jucoess.  1 2) 


Missile 


SAM  Defense 


Advanced  ACM 
Tactics 


o  Missile  firing  light 
source  lacks  accurate 
fidelity.  Need  improved 
visual  presentation. 

o  SAM  does  net  fly  profile. 
Also  need  an  expanded 
inventory  cf  "real  world" 
SAM  threat  systems. 

o  Capable  of  only  lv3l • 
Need  expanded  capability 
to  include  lvs2,  2vsl  at 
minimum. 


1-cw  Altitude  Inadequate  ground  growth. 

ACM  Need  visual  method  to 

determine  altitude  in 
simulation  cf  'real  world1 
conditions. 


Overland  ACM  o  System  is  locket!  into  an 
cverwater  male .  Need  both 
land  and  water. 


Basic  Problem  o  System  unable  to  run  de- 
Soernanoe  sired  scenarios  due  to 

unreliability  and  inade¬ 
quate  design  attention  to 
this  area. 


o  System  reconl  feature 
limited  to  1  hour.  3- 
hour  minimum  needed  to 
ccver  all  desired 
scenarios. 


Engagement 
■dearset  ry 
Arva  lysis 


o  Threat  Ixjgey  cannot  Ine 
saerviriaized  (directed)  to 
demonstrate  class  ion  1 
adversary  maneuvers.  Need 
cajnability  to  provide  this 
important  training  re- 
qu*  renunt 


Sortie  Debrief  o  System  employs  vholly  in¬ 
adequate  ri  ontvence 
measurement  .system. 


A  semtsdvat  ilifferent  system  also 
utilized  for  fighter  tactics  training  exhi¬ 
bits  similar  short  carings :  ^  ' 


o  Inadequate  instruct ional  .iesign 
futures.  Deni?  and  debrief 
systems  invioruately  designed  and 
Ufficvilt  to  use. 
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o 


Adaptive  maneuver  logic  (of  threat 
aircraft)  varies  in  performance. 

o  Demo  create  process  involves  a 
lengthy  procedure  and  is  thus  dif¬ 
ficult  to  perform. 

o  Debrief  printouts  difficult  to 
understand  and  of  limited  use¬ 
fulness. 

It  appears  from  the  foregoing  that  al¬ 
though  technology  plays  a  part,  a  significant 
number  of  identified  problems  clearly  reside 
in  the  category  of  "insufficiently  defined 
mission  and  training  requirements . " 

STR'JCTURED  APPROACH  TO  FORMULATING 
SIMULATOR  DESIGN  CRITERIA 

The  means  of  translating  appropriate 
"mission  logic"  into  full  mission  simulator 
design  features  suffers  from  a  clear,  well 
defined  process.  The  need,  for  instance,  of 
having  an  unambiguous  and  oonprehensive 
understanding  cf  the  cognizant  mission  prior 
to  addressing  conceptual  design  options  is 
self-evident.  While  it  would  be  reive  to 
suggest  that  design  teams  do  not  design  with 
the  overall  mission  in  mind,  experience  has 
shown  that  obese  &.ine  teams  ere  not  provided 
with,  and  thus  do  not  adequately  address, 
critical  mission  requirements  as  part  of  the 
conceptual  design  process.  Approaches  have 
been  utilized  in  the  past  to  attempt  to 
understand  and  document  mission  driven 
training  requirements,  yet  an  acceptable 
methodology  unfortunately  is  not  evident. 

(The  ISD  molel  is  a  movement  in  this  direc¬ 
tion  but  significant  slvortcomings  exist  for 
many  purposes  beyond  curriculum  development, 
due  to  its  inability  to  adequately  analyze 
the  cognizant  mission:  Most  of  the  method¬ 

ology  is  directed  to  specifying  tasks  and 
behaviors.)  One  possible  method,  suggested 
here,  is  present ed  in  the  interest  of  stimu¬ 
lating  dialogue  with  a  goal  of  eventually 
establishing  a  structured  approach  to 
advanced  simulator  design.  Hus  pro^xosed 
scheme,  presented  in  the  broadest  of  brush 
strokes,  follows  a  straightforward  logic  pro¬ 
cess  outlined  below  (and  graphically  dis¬ 
played  in  Figure  1).  A  more  detailed  des¬ 
cription  of  each  element  of  the  scheme 
follows  in  subsequent  paragraphs . 

Structured  Approach  to  Simulation  Design 

o  Cot* loot,  a  formal  Mission  Analysis, 
structuring  tlie  mission  to  include 
functional  sequencing,  scenu  i-os, 
task  requirements  and  jx»rforrnnee 
objectives. 

o  Perform  on  Instruct ional  System 
Roraire-.a.nts  Analysis  in  order  to 
select.  .qipropnate  instructional 
metfnds  for  selected  mission  ubjec- 
t ives . 

o  Utilizing  systematic  an’rcueh. 


Figure  1 


conduct  a  Priority  Analysis  by 
assigning  priorities  to  specific 
mission  and  rnerator  task  areas; 
spotlighting  candidate  simulation 
design  features  vdiich  will  realize 
the  greatest  training  payoff. 

Mission  Analysis 

A  selected  operational  mission  is  for¬ 
mally  analyzed  by  following  the  logic  process 
depleted  in  Figure  2.  Although  space  does 
not  permit-  a  detailed  examination  of  the 
methodology,  the  following  out lines  the  im¬ 
portant  steps. 

o  The  pro  ess  of  analyzing  a  cogni¬ 
zant  mil itary  mission  oarmences 
with  a  oiierent  -  albeit,  generic  - 
mission  statement.  Amission 
statement  Ascribes,  in  broad 
terns,  the.  purpose  of  a  warfare- 
spec  i  f  ic  <  a  erat  ion . 

o  Functional  mission  hierarchies  -are 
constructed  next  to  identify  mis¬ 
sion  element  >,  specifying  Iaroad 
mission  tasks,  and  depiction  rela¬ 
tionships  . 

o  0*erall  miss  ion  objectives  .ure  tlen 
specified  for  ippropriate  mission 
elements  identifie>l  in  tie  step 
at ove .  Objectives  are  needed  to 
understand  broad  performance  re- 
uuiremea's  of  tie  scrutinized 
mission. 


85 


MISSION  ANALYSIS 


Figure  2 


o  Scenarios  are  developed  to  explore 
threat-irpcsed  constraints,  mission 
profiles,  tactics,  critical  mission 
segments  and  bread  resource  needs 
to  successfully  carry  out  the  in¬ 
tended  operation.  Functional 
sequencing,  scenario  tree  analysis, 
and  Pfc  (Probability  cf  Kill)  eola¬ 
tions  are  the  three  oermen  analyti¬ 
cal  teduiiques  utilized  in  this 
step. 

o  Mission-task  requirements,  the  last 
step  in  the  process,  constructs  task 
hierarchies,  specifies  task  dimen¬ 
sions,  identifies  critical  decision 
points  and  relative  task  loading, 
and  sugges t s  task  performance  cri¬ 
teria  in  an  attempt  to  fully  under¬ 
stand  detailed  requirements  of  the 
mission. 

Ins  tract  iot  vi  1  System  Requirements 

Analysis 

An  analysis  of  training  requirements 
driven  the  previously  outlined  technique 
for  analyzin.  've  mission,  is  designed  to 
specify  qppre  at.e  training  tracks  and 
training  objectives  and  relate  these  to  can¬ 
didate  training  nvthods .  A  convenient  means 
to  conduct  this  a/ia lysis  is  to  create  a 
matrix  as  adwwn  in  Figure  3.  A?  indicated, 
this  rather  straightforward  analytical  tech¬ 
nique  specifies  mission  objectives  (derived 
from  the  aforementioned  miasicin  analysis)  cn 
the  horizontal  axis  vs.  candidate  training 
methods  along  the  vertical  axis.  Note  the 


ACM  portion  of  the  fighter  mission,  selected 
for  this  exanple,  is  decomposed  to  a  level 
sufficient  for  analysis  and  is  arranged  in  a 
systematic  sequence.  Finally,  for  each  mis¬ 
sion  objective,  candidate  methods  of  training 
are  identified  by  X-ing  in  appropriate  boxes. 
For  this  paper  vae  are  concerned  primarily 
with  simulation,  although  this  technique, 
being  generic,  appears  to  be  suitable  for  a 
wide  range  of  instructional  needs. 

Priority  Analysis 

After  candidate  training  methods  are 
identified,  determining  candidate  design 
areas  based  ipon  the  examination  of  critical 
mission  requirements  is  considered  the  next 
logical  step.  This  suggested  process  or 
methodology  is  outlined  below. 

o  Select  an  appropriate  mission 

objective  (or  phase)  where  simula¬ 
tor  training  is  suggested  from  the 
aforementioned  matrix. 

o  Determine  relative  importance  of 
this  mission  phase  utilizing  expert 
interviews  and  conventional  scaling 
techniques  (a  one- to- ten  scale 
appears  appropriate ) . 

o  Decanpose  ear.  c'r.ed  mission  phase 
into  its  component  tasks  (task 
hierarchy) . 

c  Arrange  related  tasks  into  groups 
or  nodules. 

o  Examine  ova  rail  difficulty  of  these 
task  mvjiules  by  determining  air¬ 
crew  task  loading  during  this  {base 
of  the  missicr  as  follows:* 

-Tasks  performed  inriivid’jally  in 
sequence  -  low  rating  (1-3) 

-Multiple  tasks  performed  simul¬ 
taneously  under  relaxed  time 
conditions  -  medium  rating  (4-6) 

-Multiple  tasks  performed  ainui- 
taneouuly  under  increased  time 
egression  -  high  rating  (7-10) 

o  Plot  mission  importance  vs  task 
difficulty  on  conventional  graph  ns 
shown  in  Figure  4. 

o  Superirrpose  demarcation  lines  <n 
the  graph  formiivj  ftxir  equal 
quadrants  . 

Mission  'iqeeti ve/task  modules  plots 
vtaich  fall  in  tv>e  right  q.vtdrant  repre¬ 

sent  the  highest  jjs  iority  areas  where  design 
effozts  should  be  foeuse* i  ani  where  maximum 

frevuxis  mi us ton  analysis,  having  iden¬ 
tified  critical  <V?cisian  joints  ard  taak 
loading  will  aid  this  analysis  considerably. 
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INSTRUCTIONAL  SYSTEMS  REQUIREMENTS  ANALYSIS  MATRIX 
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mission  ef fectivene3S  will  most  likely  be 
achieved.  Conversely,  plots  Miich  fall  in 
t he  lower  left  tjuadrant  represent  the  lowest 
priority  rating  suggesting  not  more  than  a 
mcxlest  investment  in  resources  is  needed. 

An  example  to  illustrate  the  above  scheme 
fol lows. 

From  the  Instruct iorvil  Requirements 
analysis  nvatr i x  (created  in  part  from  the 


previous  mission  analysis):  (1)  it  is  deter- 
mined  that  the  mission  objective  "Determine 
Attack  Plan"  is  a  candidate  for  advanced 
simulation  training:  and  (2)  from  expert  in¬ 
terviews  it  is  then  decided  that  this  mission 
objective's  relative  importance  rating  is  8. 
This  nurnber  is  plotter!  along  the  Y  axis  of 
the  graph  (Figure  4).  (3)  The  mission  phase 
is  then  tieounposa!  into  its  oarpanent  tasks 
and  a  task  molule  Determine  Appropriate  1V1 
Tactics  to  Achieve  Advantage  is  identifier! 
(Figure  5J.  (4)  Determining  the  difficulty 

of  this  task  module  is  then  atterrpted  by  ex¬ 
amining  task  loading  for  this  mission  phase. 

In  this  hypothetical  case  multiple  tasks 
need  to  be  performed  but  irvder  relaxed  time 
conditions  (largely  because  this  objective 
occurs  prior  to  engagement);  thus  a  dif¬ 
ficulty  ratinq  of  6  is  assignor!.  This  nuniier 
is  then  plotted  along  the  X-axis  (see  Figure 
4 ) .  Where  tliese  two  nurbers  intersect  repre¬ 
sent  s  the  relative  desirability  rating  of 
providing  instruction  for  this  mission- 
related  task  noiule  ur.il  izi.ig  c».5vaijceG  siirsu- 
letion  technology.  M>te  in  this  exarrple  it 
is  considered  "highly  desirable"  to  utilize 
advanced  simulation  systems  to  train  in  this 
area. 


The  foregoing  structured  approach  is  in¬ 
tended  to  provide  a  convenient  means  of 
specifying  and  selecting  area'.  Where  nor.- 
centrat&l  efforts  can  achieve  the  greatest 
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training  ret um-on- investment .  In  other 
words,  of  all  of  the  mission  areas  that  are 
possible  to  simulate  today,  vhere  should  we 
be  directing  our  attention?  Clearly,  cur 
attention,  followed  by  our  resources,  should 
first  be  directed  at  the  most  difficult/most 
important  mission  areas.  In  this  way,  train¬ 
ing  systems  can  be  designed  that  speak 
directly  to  critical  operational  needs. 
Finally,  it  is  the  author's  opinion  that  as 
our  high-technology  training  systems  beg’n  to 
focus  on  critical  operational  needs,  their 
acceptability  as  well  as  their  utilization 
rates  will  show  substantial  improvements . * 

INSTRUCTIONAL  CESIGN  FEATURES 

The  most  obvious  instructional  design 
feature  an  advanced  simulator  could  possess 
is  a  formal  curriculum  designed  to  achieve 
specific  training  objectives  vhile  utilizing 
the  subject  device.  Unfortunately,  this  is 
so  obvious  that,  paradoxically,  it  has  teen 
often  over  looked.  Numerous  instances  can  be 
cited  where  expensive,  advanced  flight  simu¬ 
lators  were  delivered  to  the  fleet  without  a 
supporting  syllabi.  Fortunately,  steps  are 
being  taken  to  rectify  this  situation  and  it 
appears  now  that  most  cognizant  individuals 
agree  on  a  need  for  a  supportive  instruc¬ 
tional  program  o'.rplete  with  performance 
feedback  for  these  modem  training  devices. 


*It  is  recognized  that  ultimately  this 
priorization  analysis  rrust  find  a  logical 
resting  place,  such  as  a  direct  aid  to 
selecting  generic  design  features  of  the 
system  under  analysis.  Additional  work  is 
needed  in  this  area  and  formal  support  for 
this  and  similar  efforts  is  strongly 
encouraged. 


Significant  difficulties,  however,  repain  in 
deciding  who  should  be  responsible  for  curri¬ 
culum  develcpnent:  prime  cent  rectors,  squad¬ 
ron  personnel  ( i .e , ,  users)  or  training 
specialists.  These  questions  be cane  moot 
when  these  devices  me  viewed  fran  a  systems 
perspective.  This  view  holds  that  earplete 
instruct icnal  systems  are  needed  in  this  a ne 
of  oarplex  rnissi  cr  requirements  driven  by 
high  technology. 

For  advanced  simulators  this  "training 
system"  could  include  such  operational  items 
as: 


-  Conventional  system  hardware  and 
supporting  computer  software 

-  Operational  training  curriculum 

-  Performance  measurement  system 

-  Training  nanagement  system. 

There  remains  an  unfortunate  tendency  - 
possibly  due  to  a  less  than  clear  view  of  the 
"systems  approach"  -  to  treat  only  the  first 
step  as  sufficiently  worthy  of  the  design 
team's  attention.  If  can  at  least  ten¬ 
ts  ively  agree  that  as  sophisticated  instruc¬ 
tional  devices,  full  mission  simulators  ought 
to  cone  equipped  with  a  structured  training 
program,  (which  has  the  added  benefit  of 
requiring  contractors  to  more  fully  consider 
operational  training  problems  and  require¬ 
ments)  then  an  important  related  condition 
presents  itself.  How  can  critical  mission 
objectives  lie  translated  into  formal  curricu¬ 
lum  designed  to  enhance  training  effec¬ 
tiveness  in  today's  oarplex  mission  areas? 

To  answer  this  question  we  mist  first 
briefly  outline  the  ISD  process  and  then  show 
how  the  previous  priority  scheme  can  be  uti¬ 
lize!  for  the  (design  of  enhanced  training 
programs . 

Curriculum  Development 

Development  of  a  training  program  for 
advanced  flight  simulators  should  clearly 
follow  the  ISD  (Instructional  System  De¬ 
velopment)  process,  vhich  provides  a  systema¬ 
tic  and  scientific  methodology  for 

o  Identifying  tasks  that  (operators) 
must  perform 

o  Specifying  the  hierarchy  of 
trainiiig  objectives 

o  Developing  procedures  and  schedules 
for  training  in  the  performance  of 
identified  tasks. 

The  ISD  model,  although  providing  for 
improved  professionalism  in  instructional 
system  design,  lacks  a  mechanism  specifically 
designed  to  spotlight  critical  mission  areas 
and  concomitant  priority  training  require¬ 
ments.  Thus,  such  critical  areas  as  operator 


decis  un-task  requirements  are  largely  neq- 
leeted  in  present  ISD  treatments.  (3) 

Mechanism  to  Aid  in  Development  of  Enhanced 
Training  Products 

Tb  aid  in  the  development  of  enhanced 
training  products,  a  miss ion- focused  priority 
scheme  for  identifying  critical  mission  areas 
and  thus  important  training  requirements  is 
considered  essential.  The  previously  out¬ 
lined  "structured  approach"  is  considered  a 
candidate  means  to  accomplish  this  task. 

The  process  of  focusing  in  on  critical 
mission  and  training  needs  oormences  as  be¬ 
fore  with  a  formal  analysis  (Figure  2)  of  the 
cognizant  mission.  The  Instructional  System 
Recrui rement  Analysis  matrix  (Figure  3)  then 
provides  a  convenient  means  to  identify  can¬ 
didate  instructional  methods  appropriate  to 
the  specified  mission  objectives  From  this 
matrix,  we  can  see  for  instance,  that  the 
mission  objective  "determine  threat  capabili¬ 
ties"  can  be  addressed  by  utilizing  cny  or 
all  of  the  following  training  methods: 

-  Part  task  trainer 

-  CAI 

-  Stand-alone  tape/slide /video 

-  Programmed  text 

-  Platform. 

A  possible  way  to  determine  the  most 
appropriate  instructional  strategy  is  to  con¬ 
duct  a  Priority  Analysis  to  determine  rela¬ 
tive  importance  of  this  mission  area.  This 
analysis,  explained  in  detail  in  the  previous 
section,  examines  appropriate  mission  objec¬ 
tives  and  its  oorrponent  tasks  from  the  stand¬ 
point  of  importance  and  task  difficulty. 

When  plotted  cn  the  Priority  Analysis 
Graph  (Figure  4),  Mission  Oojective/Task 
Modules  which  fall  in  the  high  priority 
quadrant  should  drive  the  development  of  an 
instructional  system  carmens  urate  with  this 
priority.  High  priority  areas  may,  there¬ 
fore,  retire  tire  develonment  of  a  corrposite 
training  systan  embracing  tire  following 
areas: 

o  Aircraft  operations  complete  with 
detailed  scenarios 

o  Simulation  training  events  designed 
to  jrrepare  for  specific  aircraft 
operation  scenarios. 

o  Part -task  trainer  designed  to  in¬ 
struct  in  critical  ilecision-task 
areas. 

o  Gonputer-aided  inst ruction  designal 
to  ^levelop  requisite  background 
information. 

Thus  the  oarposite  training  system  con¬ 
cept  extending  heyond  the  orirpartmentalizol 
view  of  stand-alone  training  systems,  pro¬ 
poses,  utilizing  the  Structured  Approach", 


to  (1)  focus  on  critical  mission  areas,  and 
(2)  integrate  various  instructional  devices 
(and  instrumented  ranges)  and  curriculum  into 
a  coherent,  goal  oriented  process  whereby 
enhanced  training  indeed  becomes  a  reality. 

SUMMARY 

It  appears  self-evident  that  a  clear 
view  of  mission  objectives,  tasks,  and  can¬ 
didate  training  methods  aoupled  with  a 
priority  scheme  is  needed  in  any  successful 
design  effort  for  advanced  simulation 
systems.  Indeed,  these  concepts  are  finding 
favor  anong  more  and  more  individuals  such  as 
Cream' who  encourages  a  task  rating  system 
for  use  in  selecting  actual  simulator  design 
features . 

Whatever  priority  or  rating  system  is 
adopted,  providing  design  teams  with  appro¬ 
priate  "mission  logic"  coupled  with  a  mecha¬ 
nism  to  focus  attention  and  resources  on 
those  areas  promising  the  greatest  training 
retum-on- investment  appears  to  be  the  criti¬ 
cal  task  at  hand.  No  longer  can  we  afford  to 
build  high  technology  systems  exclusively 
from  a  technical  speci f ication  document. 
Technical  kncw-hcw  must  be  translated  into 
mission  capability  by  making  technology 
respond  specifically  to  critical  mission 
needs.  Otherwise  we  will  remain  in  our 
current  predicament  of  fielding  technical 
systems  tiiat  at  best  cnlv  achieve  sub-optimum 
levels  of  mission  performance  throughout  the 
system's  life-cycle.  Thus  future  distress 
calls  from  the  operational  omimunity  will 
become  an  increasingly  regular  and  tragic 
occurrence . 
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ABSTRACT 

•‘•This  paper  introduces  a  candidate  needs  assessment  model  that  has  unique 
potential  for  identifying  military  training  needs.  During  the  training 
planning  process,  planners  freouently  and  inadvertently  confuse  training 
strategies  (means)  with  training  results  (ends).  One  reason  for  this  confusion  is  the 
inherent  difficulty  in  accurately  specifying  military  training  needs.  The 
complex  nature  of  military  training  needs  is  explored  and  the  potential 
benefit  of  the  Organizational  Elements  Model  as  a  planning  tool  is  presented.^ 

Quality  military  training  is  a  necessary  but  The  main  concern  of  the  OEM  is  the  identifica- 

difficult  objective  to  achieve.  The  act  of  bring-  tion  of  organizational  needs.  Kaufman^3'  defines 
ing  together  all  the  necessary  ingredients  of  a  need  as  a  gap  between  "what  is"  and  "what  should 

training  systems  (e.g.,  people,  training  aids,  be"  in  terms  of  organizational  results.  Defining 

devices  and  curricula)  will  not  insure  effective  a  need  as  a  gap  between  present  and  desired  results 

training.  More  fundamentally,  effective  training  helps  to  ensure  that  no  solution  statements  are 

is  the  result  of  careful  planning  which  is  based  included  within  the  statement  of  the  needs.  In 

upon  accurate  need  assessments.  This  paper  order  for  effective  planning  to  occur,  it  is 

examines  the  nature  of  military  training  planning  critical  that  solutions  for  the  need  not  be  intro- 

from  a  wide  perspective  with  special  attention  duced  too  early  in  the  planning  process.  Consider- 

focused  upon  the  unique  characteristics  of  ing  specific  solutions  before  the  need  is  truly 

military  training  needs.  The  principle  theme  defined  often  causes  the  planner  to  establish 

’s  that  the  planning  processes  presently  in  use  biases  and  to  overlook  many  alternative  solutions 

can  be  improved  by  using  a  relatively  new  needs  For  example,  if  a  specific  solution  is  incorporated 

assessment  technique,  thus  potentially  increas-  too  early  (i.e.,  "what  we  need  is  a  computer¬ 
ing  the  probability  that  effective  training  assisted  instruction  (CAI)  system")  the  planning 

systems  will  be  produced.  will  then  proceed  under  the  assumption  that  CAI 

is  indeed  the  only  appropriate  solution  to  the 

A  NEEDS  ASSESSMENT  APPROACH:  THE  problem.  However,  had  planning  proceeded  without 

ORGANIZATIONAL  ELEMENTS  MODEL  the  assumption  about  CAI,  the  requirement  may  have 

been  met  with  an  instructional  delivery  system 

The  Organizational  Element  Model  (OEM)  for  which  is  less  expensive  or  more  appropriate  than 

educational  system  planning,  while  currently  in  CAI  (e.g.,  a  modularized  slide-tape), 

the  developmental  stage,  is  one  method  of  assess¬ 
ing  needs.  (Ut  (2)*  The  concepts  inherent  to  the  By  viewing  the  need  as  the  difference  between 

OEM  apply  to  all  needs  inside  or  outside  of  any  "what  is"  and  "what  should  be",  solution  state- 

organization.  Hence,  the  OEM  will  first  be  nents  can  be  clearly  separated  from  needs  *tate- 

discussed  in  general  training  and  educational  ments.  This  separation  allows  the  user  of  the 

terms,  followed  by  specific  application  to  the  OEM  to  explore  all  aspects  of  the  problem  with  an 

military  training  community  (i.e.,  all  training  unprejudiced  outlook, 

commands,  their  supporting  commands,  all  con 

tractor  organizations  that  support  training  The  OEM,  which  is  composed  of  five  elements, 

efforts,  etc.).  is  the  tool  which  relates  needs  assessment  to 

other  aspects  of  training  development,  implemen..:- 
tion,  and  evaluation  (see  figure  1).  The  first 
element  represents  the  inputs  which  the  organiza¬ 
tion  receives  and  uses  in  accomplishing  its  objec- 
Mt  should  be  noted  that  the  discussion  of  the  tives.  These  inputs  consist  of  the  existing  "raw 

OEM  reoresents  a  synthesis  of  Kaufman's  writing  materials"  which  are  available  fer  use.'*' 

on  the  subject.  Only  primary  references  are  Examples  include:  expertise,  time,  funds,  corn- 

noted  in  the  text.  puters,  policies,  plans,  goals,  and  people. 
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The  second  element  of  the  OEM  consists  of 
organizational  processes.  These  processes  con¬ 
sist  of  the  means  or  solution  steps  which  start 
the  inputs  into  action.  Examples  of  orgarriza- 
tional  processes  are:  management  by  objective, 
development  of  CAI,  the  systems  approach, 
analyses,  research,  and  the  Program  Evaluation 
and  Review  Technioue. 

The  f.rst  two  elements  represent  organiza¬ 
tion  efforts.  People  within  organizations  spend 
most  of  their  time  gathering  inputs  and  using 
them  in  processes.  These  first  two  elements  of 
the  OEM  (efforts)  are  used  to  produce  the  last 
throe  elements  which  are  organizational  results. 

The  third  element  of  the  OEM  consists  of 
organizational  products.  These  are  the  "things" 
which  the  organization  produces.  These  products 
could  be  training  materials  of  various  kinds 
( e . g . ,  training  devices,  curricula,  evaluation 
reports,  etc.). 

The  fourth  element  consists  of  the  outputs 
of  the  organization.  These  are  defined  as  that 
which  is  delivered  to  the  world  external  to  the 
training  community;  such  as  trained  personnel 
(graduates).  Products  and  outputs  differ,  in 
that  an  organization's  products  are  only  inter¬ 
mediate  results  of  an  organization's  efforts. 

For  example,  the  training  conmunity  never 
delivers  curricula  to  the  external  world. 

Instead  it  uses  the  curricula  internally  to 
train  personnel.  These  trained  personnel  are 
the  outputs  which  the  training  community 
delivers  to  tho  external  world. 


The  final  and  most  important  OEM  element  is 
outcomes  of  the  organization.  Societal  outcomes 
are  nade  up  of  the  effects  or  impacts  of  the 
organization's  outputs.  For  example,  less  crime 
is  reported  in  society  because  the  graduates  of 
the  educational  establishment  were  better  trained 
and  thus  were  better  able  to  obtain  gainful  employ¬ 
ment.  In  this  example,  reduced  crime  is  one 
outcome  of  better  trained  graduates.  As  noted 
above,  the  graduates  are  outpi ts  of  the  educa¬ 
tional  system. 

Functioning  of  OEM  in  a  Training  Setting 

The  functioning  of  the  OEM  begins  with  the 
establishment  of  a  data-base  by  training  system 
users  and  developers.  Figure  2  shows  that  the 
OEM  has  two  dimensions:  "what  is"  and  "what 
should  be".  This  data-base  contains  information 
about  the  training  system,  as  it  presently  exists, 
and  also  makes  statements  about  what  the  training 
system  should  be  like.  The  data  b^se  is  used  for 
the  conduct  of  a  discrepancy  analysis  comparing 
the  "what  is"  status  fer  each  OEM  element  with 
the  "what  should  bo"  status.  The  two  dimensional 
matrix  in  figure  2  also  suggests  the  sequence  for 
conducting  this  analysis.  The  sequence  estab¬ 
lished  for  the  "what  is"  status  for  each  element 
begins  with  the  inputs  elements  and  progresses 
to  the  outcomes  element.  The  sequence  for 
establishing  the  "what  should  be"  status  is  just 
the  reverse.  Experience  has  shewn  that  the 
sequence  fo»*  analyzing  "what  should  be"  is 
neces:ary  for  determining  the  proper  linkages 
between  the  five  elements.'^'  Again,  when 
planners  consider  inputs  and  processes,  before 
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outcomes  have  been  defined,  they  run  th  risk  of 
overlooking  many  possible  solution  alternatives. 

The  cross-hatched  arrows,  represent  the  relation¬ 
ship  between  "what  is"  and  "what  should  be"  for 
both  the  inputs  and  processes  elements.  These 
arrows  indicate  that  the  discrepancies  are  in 
efforts,  not  results. 

Once  the  "what  is",  "what  should  be"  condi¬ 
tions  are  known,  discrepancies  between  internal 
and  external  results  can  be  determined.  The  OEM 
makes  a  distinction  between  results  that  are 
internal  to  the  organization  and  results  that 
are  external  to  the  organization  Examples  of 
internal  results  from  a  tank  production  scenario 
are:  obtaining  engines,  obtain  all  sub-assemblies, 
obtain  orders.  Examples  of  external  results  from 
the  tank  production  scenario  are:  tank  safety, 
efficiency,  profitability,  positive  military 
impact  and  repeat  orders. 

The  following  hypothetical  example  serves  to 
illustrate  this  process.  The  outcome  problem  can 
be  summed  up  in  the  following  statement,  "NATO 
Dragon  gunners  are  not  proficient  marksmen."  The 
hypothetical  "what  is"  status  for  each  OEM  element 
is  as  follows:  (see  figure  2) 

inputs.  Present  funding  levels  for  Dragon 
training;  number  of  instructors; 
number  of  practice  rounds;  ranges; 
Dragons;  and  trainees. 

processes:  Dragon  curriculum,  practice,  quali¬ 

fication  (due  to  limited  rounds, 
the  Dragon  trainees  receive  too 
little  practice). 


products:  Skills,  knowledge  and  attitude 

necessary  to  become  proficient  Dragon 
gunners.  (Due  to  limited  practice  the 
present  training  does  not  allow  the 
trainees  to  achieve  the  necessary 
levels  of  skills,  knowledge  and 
attitudes) . 

outputs:  Only  32  of  the  current  graduates 

achieve  first  rouno  hits  at  sta¬ 
tionary  targets  400  to  600  meters 
away  in  simulated  situations. 

outcomes:  Based  on  simulated  practice  it  is  felt 
that  NATO  Dragcn  gunners  will  be  in¬ 
effective  in  actual  combat.  (This 
type  of  outcome  statement  points  up  the 
difficulty  of  assembling  rational 
"what  is"  concepts  for  the  military. 
Since,  thankfully,  so  little  live 
combat  takes  place  we  seldom  know  for 
sure  how  effective  our  outputs  are. 
Analysis  of  recent  combat  situations, 
like  the  Falklands  and  Middle  East 
conflicts,  are  of  some  help  in  this 
area). 

The  "what  should  be"  conditions  are  as 
follows: 

outcomes:  100  of  Dragon  gunners  should  be 
effective  in  the  world  external  to 
the  training  contnunity  as  measured 
by  their  ability  to  hit  95.  of  their 
targets  in  actual  combat. 
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outputs:  1 00%  of  graduates  should  prove  that 

they  have  learned  the  requisite 
skills,  knowledge  and  attitudes  as 
measured  by  th'ir  ability  to  achieve 
first  round  hits  at  *'00  to  600  meters 
in  simulated  situitijns. 

products:  The  training  community  should  develop 

training  products  which  are  useful  to 
Dragon  trainees  as  measured  by  train¬ 
ees  learning  the  requisite  skills, 
knowledge,  and  attitudes. 

The  "inputs"  and  "processes"  for  the  "what 
should  be"  are  not  defined  yet. 

The  three  'what  should  be"  elements  describe 
the  changes  which  are  required  to  meet  the  need. 
Notice  that  the  term  "as  measured  b*'"  is  included 
with  each  "what  should  be"  statement.  It  is 
important  that  the  criterion  benchmarks  for 
success  be  established  early  in  the  planning 
process.  It  is  also  important  to  note  that  there 
are  no  specific  solutions  presented.  Solution 
alternatives  are  developed  later  when  inputs  and 
processes  are  considered.  The  inputs  and  pro¬ 
cesses  f're  concerned  not  with  what  must  be  done, 
but  rather  with  how  it  should  be  done.  In  this 
particular  exampTeT  the  processes  used  may  include 
the  formal  school  expanding  its  objectives  and 
supporting  courseware  as  well  as  incorporating  a 
training  device.  To  support  the  process  elements, 
the  inputs  should  reflect  an  increase  in  funds, 
number  of  instructors,  and  number  of  practice 
rounds. 

In  summary,  this  process  of  describing  the 
current  "whau  is"  and  the  ideal  "what  should  be" 
for  each  of  the  five  elements  ’S  potentially 
applicable  to  most  military  training  situations. 
How  the  OEM  can  assist  the  current  planning  pro¬ 
cess  is  addressed  below. 

CONFUSION  OF  ORGANIZATIONAL  MEANS  AND  ENDS 

Too  often  the  ultimate  goal  of  the  military 
training  effort  is  lost  in,  or  confused  with,  the 
multitudes  of  processes  and  products  which  are 
involved  in  the  delivery  of  a  training  system. 
These  products  and  processes  may  satisfy  the  train 
ing  community's  internal  needs  ( e . g . .  training 
courses,  training  devices,  graduates,  continued 
funding),  but  they  do  not  necessarily  satisfy  the 
proper  external  need  -  that  of  military  pre¬ 
paredness  (:.e.,  combat  effectiveness). 

A  commonly  observed  example  of  a  means-ends 
confusion  is  a  "state  ^-the-art"  training  device 
that  the  user  finds  marginally  useful  (user  in 
this  paper  refers  to  the  operational  community). 
The  term,  "state-of-the-art",  in  this  situation, 
implies  that  this  device  contains  the  latent  in 
technological  sophistication.  Yet  tMs 
sophistication  alone  does  not  ensure  user  accep¬ 
tance  or  eventual  combat  effectiveness.  User 
comments  about  the  device  may  include  complaints 
about  the  iack  of  fidelity  in  the  device's  con¬ 
trols,  confusing  feedback  displays,  and  worst  of 
all,  the  observation  that  the  device  does  not 
assist  in  producing  trained  personnel.  The 
problem  is  that  the  device's  sophistication 
applies  only  to  its  technological  components 
(the  means  for  delivering  training)  and  not  to 


how  they  function  to  oroduce  combat  effectiveness. 

Although  the  means  vs.  ends  confusion  in  the 
planning  process  has  many  causes,  this  paper  will 
address  only  two.  The  first  reason  is  the  mis¬ 
placed  emphasis  upon  internal  results  while  eschew¬ 
ing  user  concerns  (external  results).  The  second 
reason  for  the  means  vs.  ends  confusion  is  the 
inherent  difficulties  in  defining  military  training 
needs.  The  nature  of  these  causes  and  how  the  OEM 
can  assist  planners  in  redjcing  the  means  vs.  ends 
confusion  are  developed  below. 

Training  organizations  often  and  inadvertently 
emphasize  internal  results  as  opposed  to  external 
results.  That  is,  training  organizations  often 
derive  their  own  internal  objectives  (e.g.,  receiv¬ 
ing  money,  developing  training  devices,  graduating 
students,  etc.)  and  pursue  them.  Such  internal 
objectives  are  important;  they  alww  ^anizations 
the  opportunity  to  determine  progress  -os  an 
ultimate  end  However,  problems  occur  wnen  a:, 
organization  becomes  so  concerned  with  these 
internal  objectives  that  it  fails  to  properly 
identify  and  meet  external  ends.  The  training 
organization  becomes  so  focused  on  the  "how  to  do 
its"  that  it  fails  to  meet  the  larger  needs  of  the 
society  it  serves.  In  the  case  of  military  train¬ 
ing  organizations  the  external  referent  is  the 
operational  force.  Whenever  a  training  organiza¬ 
tion  becomes  so  set  on  achieving  its  own  ir.ternal 
objectives,  to  the  exclusion  of  th^  external 
society  it  serves,  detrimental  consequences  can 
and  do  occur.  Because  the  OEM  clearly  defines  the 
relationship  between  internal  results  and  external 
results,  its  use  in  the  planning  process  may  assist 
in  averting  this  cause  of  confusing  means  and  ends. 

The  second  reason  for  confusing  meais  and  ends 
in  the  planning  process  is  the  difficulty  encoun¬ 
tered  in  accurately  defining  military  training 
needs.  Specifically,  there  are  three  character¬ 
istics  inherent  to  military  training  needs  .ct 
must  be  systematically  addressed. 

The  first  characteristic  of  military  training 
needs  is  the  way  in  which  they  are  specified. 
Training  needs  usually  evolve  after  the  relations 
between  the  tl  reat,  doctrine  and  force  structures 
are  defined.  At  this  point  the  training  need  is 
usually  expressed  as  a  gap  between  the  current 
force  status  (what  is)  and  a  future  optima!  status 
(what  should  be).  This  gap  (or  need)  ir.  most  often 
expressed  in  general  terms  of  military  capa¬ 
bilities,  which  are  related  to  combat  effectiveness, 
and  military  political  impact.  The  OEM's  distinc¬ 
tion  between  training  outputs  and  outcomes  may  be 
useful  to  planners  in  coping  with  this  character¬ 
istic.  For  example,  outcomes  should  be  stated  in 
terms  of  combat  effectiveness. 

The  second  characteristic  of  military  training 
needs  is  related  to  where  the  training  requirements 
are  specified  in  the  Department  of  Defense.  Mili¬ 
tary  training  requirements  originate  from  two  basic 
sources  mostly  outside  the  training  community:  the 
parent  service  (i.e..  Air  Force,  Army,  Marines, 

Navy)  and  the  Comanders  in  Chief  of  the  unified 
commands^'-  The  parent  service  is  charged 
statutorial ly  to  provide,  among  other  items,  "Unit 
mission  proficiency  standards  for  the  conduct  and 
evaluation  of  training.  These  are  developed  from 


current  doctrine  and  force  structure.'^)  The 
unified  chain  of  command  provides  some  train¬ 
ing  requirements  when  it  states  mission  specifi¬ 
cations  and  performs  contingency  planning. 
Although  the  training  community  may  assist  in 
defining  training  needs,  it  is  the  user  commu¬ 
nity  which  must  make  the  final  decision.  Hence, 
it  is  imperative  for  the  planner  to  clearly 
define  the  nature  of  the  need  to  oc-  addressed 
in  the  tasking.  The  OEM  can  assist  the  planner 
in  determining  whether  the  tasking  is  requesting 
an  internal  result  (products,  or  outputs)  or  an 
external  result  (outcome). 

The  third  characteristic  of  military  train¬ 
ing  needs  is  that  they  experience  what  may  be 
described  as  a  translation  process  as  they  are 
passed  from  higher  to  lower  commands.  An  over¬ 
simplified  example  is  depicted  in  figure  3.  As 
noted  earlier,  a  training  need  is  defined  at 
the  highest  echelons  of  command.  Training 
needs  are  then  translated  into  concrete  program 
efforts  by  a  variety  of  conmands  subordinate  to 
the  top  levels.  These  programs  are  often  com¬ 
posed  of  several  projects.  This  translation 
process  is  not  well  understood  and  the  training 
needs  may  be  altered  in  the  extreme  from  one 
command  to  the  n^xt.  They  are  unfortunately 
omitted  entirely  in  some  translations. 


An  additional  aspect  of  the  translation  pro¬ 
cess  which  adds  r  the  confusion  is  tnat  personnel 
with  varied  backgrounds  and  technical  disciplines 
tecome  involved.  At  the  top  echelons  of  conrnand, 
the  training  need  is  discussed  in  terms  of 
abstract  military  operations  and  the  principles  of 
war.  As  the  statement  of  the  training  need  is 
passed  to  lower  echelons  it  becomes  more  concrete. 
The  five  elements  of  the  Organizational  Elements 
Model  are  considered  end  the  management  of  the 
training  system  is  defined.  At  the  project  level 
the  need  is  translated  into  the  language  of  human 
learning  and  performance,  man-machine  interfaces, 
etc.  At  the  top  levels  the  (abstract  level), 
training  needs  are  described  as  "what  must  be 
done".  As  the  needs  are  passed  down  to  t.te  middle 
and  lower  conmands  (concrete  levels)  they  are 
expressed  as  "how  to  get  it  done".  Should  this 
orderly  process  be  altered  (i.e.,  the  "how  to  get 
it  done"  is  addressed  before  the  "what  must  be 
done")  confusion  of  means  and  ends  is  likely  to 
occur. 

The  OEM's  five  elements  appear  to  have  some 
rough  correspondence  to  the  translation  process. 
For  example,  lower  echelon  efforts  may  be  classi¬ 
fied  as  inputs  and  piocess  efforts  while  higher 
echelon  concerns  fall  within  products,  outputs, 
outcomes  categories.  However,  the  "translation 
process"  r'er  se  is  little  understood  and  must 
be  investigates  before  it  can  be  adequately 
addressed  by  planners. 
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Figure  3.  The  Translation  Process:  Translating 
Operational  Needs  into  Training  Needs 
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SUMMARY  AND  CONCLUSIONS 


REFtRENCE  LIST 


Confusing  training  means  and  ends  can 
greatly  hamper  the  attainment  of  effective 
training.  The  confusion  of  means  and  ends 
can  be  eliminated  by  careful  planning  which 
is  driven  by  accurate  identification  of  user 
needs.  Planners  must  be  aware  of  how  needs 
are  stated  and  where  they  originate.  Addi¬ 
tionally,  the  process  of  translating  needs 
from  upper  to  lower  levels  of  command  must 
be  continually  addressed. 

The  "raining  community  can  improve  its 
performance  by  making  the  distinction  between 
inputs,  processes,  products,  outputs  and  out¬ 
comes  in  its  planning  efforts.  With  the  ever 
increasing  requirement  for  efficient  and 
effective  training  in  the  military,  it  is 
vitally  important  that  the  military  training 
community  understand  user  needs.  Although 
the  Organizational  Elements  Model  presented 
here  is  still  in  development,  it  is  one 
approach  to  needs  assessment  that  has  poten¬ 
tial  applicability  to  military  training. 

The  training  community  is  often  not  as 
effective  as  it  might  be  because  it  confuses 
means  and  ends  by  concentrating  more  on  pro¬ 
cesses  and  products  and  not  on  outcomes. 

This  narrow  perspective,  which  emphasizes 
the  internal  needs  of  the  training  community, 
and  often  excludes  the  external  needs  of  the 
operational  forces,  results  in  low  training 
validity.  Consequently,  disi 1 lusionment  with 
the  training  community  is  a  result.  Careful 
attention  to  user  ne^ds  will  help  to  produce 
training  which  improves  the  nation's  ability 
to  prevent  armed  host i 1 i t les  through  strength 
if  possible,  and  to  win  a  swift  and  complete 
victory  if  necessary. 
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ABSTRACT 

The  use  of  flight  simulators,  compute  r-based  instruction 
and  maintenance  training  simulators  for  training  is  evaluated 
on  the  basis  of  their  effectiveness  and  cost.  Flight  simula¬ 
tors  are  cost-effective,  compared  to  the  use  of  aircraft,  for 
training;  so  are  maintenance  training  simulators  compared  to 
actual  equipment  trainers.  Computer-based  instruction  is  as 
effective  as  conventional  instruction;  comparable-cost  data 
are  not  yet  available,  so  one  cannot  say  whether  it  is  also 
cost-effective.  These  three  methods  of  training  are  not  rare 
effective  than  the  methods  to  which  they  were  compared,  except 
for  small  improvements  in  a  few  cases.  It  is  possible  they 
could  be  made  more  effective  if  cosl^fovi  ngs  were  not  a  major 
goal,  but  this  remains  to  be  determined.  The  goal  of  analyses 
of  training  should  be  an  ability  to  perform  trade-offs  of  the 
effectiveness  and  costs  of  new  methods! of  training,  but  no  such 
trade-offs  have  yet  been  made.  I 


PURPOSE 

The  purpose  of  this  paper  is  to  eval¬ 
uate  both  the  cost  and  effectiveness  of 
military  training  in  three  areas  where 
relevant  data  are  available:  flight  simu¬ 
lators,  computer-based  instruction,  and 
maintenance  training  simulators. 

THE  MAGNITUDE  OF  MILITARY  TRAINING 

Military  training  makes  a  large  and 
continuing  demand  on  resources  allocated 
to  the  military  services.  For  example, 
the  time  spent  by  students  in  individual 
training  at  designated  schools  plus  that 
of  the  instructors  needed  at  those 
schools  accounts  for  about  one- fourth  of 
all  the  man-years  (both  military  and  ci¬ 
vilian)  available  to  the  Department  of 
Defense  (Figure  1).  About  20  percent  of 
all  military  personnel  are  in  schools  at 
all  times,  either  as  students  or  instruc¬ 
tors.  Most  (about  76  percent)  of  this 
effort  simply  provides  initial  training 
to  new  personnel  entering  military 
service  for  the  first  time. 

Individual  training  at  military 
schools  will  cost  $12.8  billion  in  Fiscal 
Year  1983.  The  types  of  training  vary 
widely  by  number  of  students  and  cost  per ■ 
student,  as  shown  in  Figure  2.  This  in¬ 
formation  can  help  tell  us  where  improve¬ 
ment  in  training  could  have  large  impact. 
For  example,  specialized  skill  training 
involves  the  largest  number  of  students 
and  costs  more  than  any  other  type  of 
individual  training.  Undergraduate  flight 
training,  for  pilots  and  other  aircrew, 
has  the  smallest  number  of  students,  but 
it  costs  much  more  per  student  than  any 
other  typo  of  individual  training. 

It  may  be  noted,  in  passing,  that  a 
student  needs  additional  training  after 


he  leaves  school.  This  is  known,  vari¬ 
ously,  as  on-the-job  training,  advanced 
flight  training,  crew  training,  and  field 
exercises.  None  of  the  costs  of  addi¬ 
tional  training  needed  after  leaving  a 
school  is  included  in  the  data  shown  in 
Figure  2. 

Finally,  it  may  help  to  know  that  the 
cost  of  individual  training  at  school  is 
nontrivial  when  compared  to  other,  readily 
available,  costs  in  the  annual  military 
budget.  As  shown  shown  in  Figure  3,  indi¬ 
vidual  training  at  school  costs  about 
half  (or  more)  of  what  is  spent  for  intel¬ 
ligence  and  communications,  the  strategic 
forces,  or  for  all  research,  development, 
test,  and  evaluation;  and  roughly  15 
percent  of  how  much  is  spent  for  procure¬ 
ment  or  operation  and  maintenance. 

Military  training  is  a  major  component  of 
the  military  budget.  Operational  readiness 
depends  on  effective  training.  It  is 
possible  to  realize  the  high  levels  of 
performance  built  into  our  advanced  weapon 
and  support  systems  only  if  these  systems 
are  maintained  and  operated  according  to 
their  design  specifications,  and  this 
also  depends  on  effective  training. 

Thus,  it  is  almost  mandatory  to  have 
effective  training;  improvements  in  our 
current  methods  of  training  are  likely  to 
have  useful  and  significant  payoffs. 

There  is,  generally,  more  than  one 
way  of  training  m?Tst  skills,  e.g.,  group 
lectures,  individual  coaching,  use  of 
simulators,  and  so  on.  The  first  question 
which  must  be  a nswered--whether  it  con¬ 
cerns  new  or  current  technology--is,  "Is 
it  any  good?  Does  it  train  as  well  as,  or 
perhaps,  better  than  some  other  way  we  - 
could  use?"  This  is  the  issue  of  effec¬ 
tiveness  of  training.  But  the  effective¬ 
ness  of  training  cannot  be  addressed  mean¬ 
ingfully  without  also  considering  its 
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MAN-YEARS  (000) 

MILITARY 

CIVILIAN 
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INSTRUCTORS 
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59 

TOTAL 

388 

59 

DoD  END-STRENGTH 

2148 

1035 

PERCENT  IN  TRAINING 

18% 

6% 

t-14-41  • 


FIGURE  1.  The  Amount  of  Time  Spent  by  Students  and  Instructors  in  Individual  Training 
at  Military  Schools,  FY  1983  (Sour;.-  Military  Manpower  Training  Report  for 
FY  1983;  Weinberger  1982). 
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FIGURE  2.  Number  of  Students  and  Costs  of  Various  Typos  of  kdhriduaJ  Training  at  Military 
Schools.  FY  1963  i?*rce.  Military  Manpower  Training  Report  for  FY  1963). 
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ITEM 

COST 

(Billions) 

INDIVIDUAL 

SCHOOL  TRAINING,  AS 
PERCENT  OF  OTHER  COSTS 

INDIVIDUAL  SCHOOL  TRAINING 

INTELLIGENCE  AND  COMMUNICATIONS 

$12.8  B 

18.0 

71% 

STRATEGIC  FORCES 

23.1 

55 

RDT&E 

24.3 

53 

OPERATION  AND  MAINTENANCE 

70.4 

18 

PROCUREMENT 

89.6 

14 

•-Mai 

FIGURE  3.  A  Comparison  ol  tho  Budgets  for  Individ  jal  School  Training  and  Othar  Major  Military 
Expenditures,  FY  1983  (Sourer  Wainbergar  1982). 


cost.  Almost  any  method  of  training  could 
be  made  effective  if  enough  money  for 
equipment  and  instructors  and  long  train¬ 
ing  time  for  students  students  were  made 
available.  Therefore,  if  two  methods  of 
training  are  equally  effective,  we  should 
select  the  one  that  costs  less.  Tf  two 
methods  of  training  cost  about  the  sane, 
we  should  select  the  one  that  is  more 
effective.  If  a  new  method  of  training 
is  more  effective  and  costs  less,  the 
choice  is  easy  and  obvious,  but  such 
opportunities  are  rare. 

We  will  consider  next  comparisons  of 
the  effectiveness  and  cost  of  three  types 
of  training:  flight  simulators  and  air¬ 
craft,  computer-based  instruction  and 
conventional  instruction,  and  maintenance 
training  simulators  and  actual  equipment 
trainers.  This  will  be  followed  by  a  dis¬ 
cussion  of  what  the  results  mean  for  future 
evaluations  of  training. 

FLIGHT  SIMULATORS* 

The  extent  to  which  flight  simulators 
should  be  used  in  flight  training  is  a 
major  concern  to  all  military  services. 

The  key  questions  are  oLvious: 

o  do  flight  simulators  really  train 
pi  lots 

o  do  the  skills  learned  in  flight 
simulators  transfer  readily  to 
ai rcraf  t 

o  are  flight  simulators  worth  what 
they  cost 

The  last  question  cannot  be  ignored  since 


^This  information  is  based  on  Orlansky 
and  String,  1977. 


the  cost  of  some  modern  flight  simulators, 
when  equipped  with  advanced  visual  systems 
and  motion  bases,  approaches  the  cost  of 
aircraft . 

The  procurement  of  new  simulators, 
including  major  improvements  to  those 
acquired  previously,  has  averaged  abou^. 

$275  million  per  year  for  the  last  eight 
years  (Figure  4).  It  costs  about  $3.6 
billion  per  year  for  fuel  and  supplies 
needed  to  operate  military  aircraft. 

About  7,600  new  pilots  will  be  qualified 
in  fiscal  year  1983  at  a  cost  cl  about 
$1.8  billion. 

How  much  does  it  cost  per  hour  to 
operate  simulators  and  aircraft?  Figure  5 
shows  comparable  operating  cost  data  for 
42  pairs  of  flight  simulators  and  aircraft 
for  fiscal  years  1980  and  1981.  Variable 
Operating  Costs,  as  shown  here,  are  for 
fuel,  oil,  and  spare  parts  consumed  as  a 
function  of  use;  the  costs  of  pay  and 
amortization  are  not  included  in  these 
amounts.  The  median  ratio  of  simulato'*/ 
aircraft  operating  costs  is  0.08;  it  was 
0.12  in  fiscal  ye<.s  1975  and  1976. 

The  fact  that  flight  simulators  cost 
less  to  operate  than  aircraft  is  a  useful 
but  not  conclusive  finding  unless  we 
also  know  how  well  they  may  be  used  to 
train  pilots.  The  effectiveness  of  a 
flight  simulator  may  be  evaluated  oy 
determining  how  much  time,  if  any,  it 
saves  in  training  pilots  to  perform  speci¬ 
fic  tasks  in  an  airplane,  compared  to 
the  amount  of  tine  required  for  such  train¬ 
ing  only  in  t,te  airplane.  The  same 
level  of  performance  in  aircraft  is  re¬ 
quired  in  both  cases.  It  has  long  been 
known  tnat  pilots  can  pt  rform  in  the  air, 
more  or  less  well,  tasks  learned  in  flight 
simulators.  The  real  question  is  how  good 
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FIGURE  4.  Major  Costs  Associated  with  All  Flight  Training 
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FIGURE  5.  Variable  Operating  Costs  Per  Hour  lor  4*  Flight  Simulators  and  Aircraft. 
FY  1980  and  FY  1981 
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is  this  training.  An  index,  called  the 
Transfer  Effectiveness  Ratio  (TER),  shows 
the  amount  of  flight  time  saved  as  a  func¬ 
tion  of  the  amount  of  time  spent  in  the 
simulator  (Figure  6). 

Transfer  effectiveness  ratios  from  34 
different  studies  of  flight  training  (e.q., 
basic  contact  fliaht,  instrument  flight, 

ASW  maneuvers,  multi-engine  transition)  are 
shown  in  Figure  7.  The  TERs  range  in  value 
from  -0.4  to  1.9,  with  a  median  value  of 
0.48.  Using  the  median  value,  the  data 
show  that  pilots  trained  in  simulators 
needed  less  time  in  aircraft  to  perform 
acceptably  than  pilots  trained  only  in 
aircraft;  the  amount  of  flight  time  saved 
was  about  one- half  of  the  time  spent  in 
the  simulator.  Tnere  is  a  wide  range  in 
these  TERs;  clearly,  these  flight  simu¬ 
lators  were  not  equally  effective  in 
saving  flight  time  in  all  cases,  and  it 
seems  irrqwrtant  to  identify  the  factors 
that  make  for  efficient  and  inefficient 
uses  of  simulators. 

Since  flight  simulators  save  about  50 
percent  of  the  time  needed  by  pilots  to 
train  in  aircraft  and  cost  'nly  8  percent 
as  much  to  use  (median  values  in  both 
cases),  it  is  clear  that  flight  simulators 
are  cost-effective,  compared  to  the  use 
of  aircraft  alone  for  training.  The  amount 
of  such  savings,  after  providing  for  the 
cost  of  these  using  simulators,  is  shown 
for  three  cases  in  Figure  8;  the  savings 
were  large  enough  to  amortize  the  procure¬ 
ment  cost  of  flight  simulators  within  two 
years  or  less. 

COMPUTER-BASED  INSTRUCTION ( 2 ) 

The  question  addressed  here  is  whether 
inst  'tion  that  is  supported  by  the  use  of 
comp,  rs  is  cost-effective  compared  to 
conventional,  classroom  instruction.  Con¬ 
ventional  instruction  is  based  largely  on 
lecture,  discussion,  and  some  amount  of 
individual  coaching.  Its  salient  features 
are  group-pacing,  and  some  amount  of  indi¬ 
vidual  attention.  Conventional  instruc¬ 
tion  was  compared  to  two  new  methods  of 
instruction,  both  using  computers,  that 
have  been  developed  since  about  1960.  The 
first  method,  Computer-Assisted  Instruc¬ 
tion  ( CAI ) ,  involves  placing  all  the  in¬ 
structional  material  in  the  computer.  It 
provides  lessons  to  the  student  by  means 
of  a  cathode  ray  tube;  the  student  may 
respond  by  such  means  as  a  keyboard  or  by 
touching  the  screen.  The  computer  guides 
the  student,  corrects  him  as  needed,  and 
provides  new  or  explanatory  material  to 
suit  his  method  of  learning;  the  computer 
maintains  student  progress  records  and  can 
provide  a  variety  of  administrative  infor¬ 
mation  needed  to  operate  a  course  and/or 
school.  In  the  second  method,  Computer- 
Managed  Instruction  (CMI),  the  student 
receives  his  instruction  away  from  the 
computer,  i.e.,  in  a  learning  carrel  or  at 
a  laboratory  bench.  However,  he  takes  a 
test  on  some  amount  of  course  material. 


"f^Based  on  Orlansky  and  String,  1979. 


typically  after  about  one  hour’s  worth  of 
instruction.  The  computer  scores  the 
test,  interprets  the  results  for  the  stu¬ 
dent,  and  directs  him  to  his  next  lesson 
or  to  repeat  the  last  one.  The  computer 
perfoims  a  similar  evaluation  and  guidance 
function  in  both  cases,  except  the t  it  is 
based  on  individual  responses  in  CAI  and 
comes  only  after  tests  in  CMI.  The 
salient  features  of  CAI  and  CMI  are  that 
each  student  proceeds  at  his  own  rate  of 
learning  ("self-paced"  instruction)  and 
receives  frequent  feedback  on  his 
progress.  This  may  be  contrasted  with 
group-paced  instruction  where  tests  are 
spaced  further  in  time  (daily,  weekly  or 
monthly)  and  where  all  students  must  pro¬ 
ceed  at  the  same  pace  despite  known  dif¬ 
ferences  in  rate  and/or  style  of  learning. 

Computer-assisted  and  computer- 
managed  instruction  are  significant 
alternatives  to  conventional  instruction 
primarily  in  specialized  skill  training. 
About  1.4  million  students  are  trained  in 
various  special  skills  at  a  cost  of  about 
$2.9  billion  each  year  (Figure  9).  Most 
(76  nercent)  of  these  students  are  "new 
accessions",  i.e.,  personnel  recently 
admitted  to  the  military  services  who  are 
being  trained  for  their  first  military 
assignment.  There  is  a  continuous  need 
to  train  replacements  for  those  who  now 
leave  the  military  services  after  relativ¬ 
ely  short  careers. 

CAI  and  CMI  may  have  many  advantages, 
but  the  fundamental  question  must  be  how 
effective  they  are,  compared  to  conven¬ 
tional  instruction,  in  instructing  stu¬ 
dents.  This  issue  has  been  addressed  in 
48  studies,  conducted  from  1968  to  1979, 
that  are  summarized  in  Figure  10.  Using 
grades  in  end-of-course  tests  as  a  measure 
t-hese  studies  compared  achievement  c: 
students  in  the  same  courses  when  taught 
by  conventional  instruction  or  by  CAI  or 
CMI.  Six  different  CAI  systems  and  two 
different  CMI  systems  were  compared  in  a 
wide  variety  of  technical  training 
courses.  Student  achievement  witn  CAI  or 
CMI  is  shown  in  the  figure  either  as 
"inferior",  "same",  or  "superior"  tu 
achievement  with  conventional  instruction 
for  the  sane  course;  the  result  of  each 
study  is  represented  by  a  dot  placed  in 
the  appropriate  column.  The  figure  shows 
clearly  that  student  achievement  at  school 
with  CAI  or  CMI  is  about  the  same  as,  or 
in  some  cases  superior  to,  that  with  con¬ 
ventional  instruction.  The  amount  of  supe 
rior  achievement  (i.e.,  higher  scores) 
is  not  large. 

In  conventional  instruction,  the 
rate  at  which  new  information  is  pre¬ 
sented  is  based  on  about  how  much  the 
average  student  can  absorb.  The  entire 
class  proceeds  at  the  same  rate  (called 
"group-pacing"  or  "lock-step"  instruc¬ 
tion);  ♦•his  pace  necessarily  holds  back 
student  who  can  learn  at  a  faster  rat  . 
With  CAI  and  CMI,  each  student  proceeds 
at  his  own  rate.  Thus,  we  should  expect 
that  students  instructed  by  CAI  or  CMI 
should  complete  their  courses  in  less 
time,  on  the  average,  than  those  taught  by 
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FIGURE  7.  Transfer  Effectiveness  Ratios  for  Various  Types  of  Flight  Training.  Based  on  22  Studies  Performed 
1967-1977. 
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conventional  instruction.  Data  on  student 
time  savings,  drawn  from  the  studies  on 
student  achievement  reported  above,  are 
shown  in  Figure  11.  The  average  amount  of 
student  time  saved  by  CAI  or  CMI ,  compared 
to  conventional  instruction,  shows  a  very 
wide  range — from  -30  percent  to  as  much  as 
85  percent  (i.e.,  in  a  few  cases,  CAI  or 
CMI  took  more  time).  The  median  value  is  a 
time  saving  of  about  30  percent. 

There  are  few  data  that  compare  the 
costs  of  C*I  or  CMI  and  conventional  in¬ 
struction  for  the  same  course,  and  these 
comparisons  tend  to  be  incomplete.  Data 
on  student  time  savings  appear  to  be  reli¬ 
able-?  however,  many  other  costs  must  also 
be  considered,  e.a.,  for  program  develop¬ 
ment  (hardware,  course  materials,  program¬ 
ming),  the  delivery  of  instruction  (in¬ 
structors,  support,  supplies,  repairs), 
and  student  pay  and  allowances.  No  such 
comparison  was  made  in  any  of  the  studies 
reported  above  that  contained  data  on  stu¬ 
dent  achievement  and  time  saved. 

It  is  often  suggested  that  computer- 
based  instruction  is  cost-effective  be¬ 
cause  the  cost  of  computers  has  declined 
dramatically  in  the  last  few  years.  This 
true  statement  can  be  misleading  because 
it  overlooks  the  fact  that  the  costs  of 
developing  software  and  course  materials 
(both  labor-intensive)  have  been  increas¬ 
ing  over  the  same  time  period.  Some  stud¬ 
ies  on  the  cost-effectiveness  of  the  PLATO 
IV  system  (at  Aberdeen  Proving  Ground, 

North  Island,  and  Chanute  Air  Force  Base) 
and  the  Air  Force  Advanced  Instructional 


System  (at  Lowry  Air  Force  Base;  were 
conducted  during  the  period  of  1975  to 
1978.  The  general  findings  were  that, 
compared  to  conventional  instruction, 
these  systems  were,  at  that  time,  either 
marginally  cost-effective  or  not  cost- 
effective.  None  of  these  studies  can  be 
represented  as  conclusive  or  used  as  a 
basis  for  judgment  at  the  present  time 
because  of  the  large  improvements  in  com¬ 
puter  technology  that  have  taken  place 
since  they  were  completed. 

Thus,  it  can  be  said  that  computer- 
based  instruction  is  as  effective  as  con¬ 
ventional  instruction  on  the  basis  of 
student  achievement  at  school,  and  that 
it  saves  student  time  needed  to  complete 
various  courses.  However,  comparisons  of 
the  total  costs  of  these  methods  methods 
of  instruction,  using  current  data  and 
technology,  remain  to  be  accomplished. 

MAINTENANCE  TRAINING  SIMULATORS < 3 ) 

Operational  equipment  is  used  often 
at  schools  to  train  technicians  how  to 
perform  routine  maintenance,  diagnose 
malfunctions,  and  replace  faulty  parts. 

It  seems  reasonable  to  do  this  because 
trainees  will  have  to  maintain  the  same 
equipment  later  in  the  field.  In  addition, 
actu  j1  equipment  needed  for  training  can 
be  acquired  readily  by  buying  an  additional 
unit  off  the  production  line.  Unfortu¬ 
nately,  the  use  of  actual  equipment  is  not 

(3*Based  on  Orlansky  and  String,  1981. 


necessarily  the  most  effective  way  to 
train  new  technicians.  The  instructor  must 
be  able  to  insert  "faulty"  parts  in  order 
to  demonstrate  fault  finding  procedures. 
Apart  from  the  fact  that  it  takes  some 
time  to  prepare  each  lesson  on  faults, 
there  will  be,  at  best,  a  limited  reper¬ 
toire  of  faults  that  can  be  illustrated. 

Wear  and  tear  on  "failed"  parts  makes  them 
easy  to  detect  with  little  regard  to  the 
particular  malfunction  that  is  being 
demonstrated.  Major  faults  or  casualties 
are  not  introduced  in  large  systems  where 
the  investment  cost  is  high  for  fear  of 
damaging  the  trainer.  For  the  same  reason, 
only  the  instructor  makes  cal ibrat ional  or 
adjustment  changes.  Cascading  casualties 
are  difficult  if  not  impossible  to  induce 
into  real  equipment.  Further,  it  is  dan¬ 
gerous  to  use  some  operational  equipment 
in  a  school  because  of  high  voltages  or 
pressures;  operational  equipment  may 
break  down  at  school  because  of  abuse  by 
students,  not  be  ready  when  needed  fcr 
training,  and  may  entail  high  maintenance 
costs. 

Most  of  these  limitations  may  be 
overcome  by  using  maintenance  training 
simulators,  rather  than  actual  equipment, 
to  train  technicians.  Such  simulators 
always  incorporate  some  type  of  computer- 
based  control.  Thus,  it  is  easy  to  provide 
a  large  number  of  malfunctions  (e.g.  ,  com¬ 
binations  cf  instrument  readings  and  non- 
responsive  controls)  that  an  instructor 
can  eisily  and  quickly  select.  Simulators 
can  be  made  both  safe  for  use  and  resistant 
to  abuse  by  students.  Above  all,  being 
computer-based,  such  simulators  can  record 
student  performance  and  provide  knowledge 
of  results  that  otherwise  would  require 
fulltime  observation  of  every  student  by  a 
qualified  instructor.  The  use  of  mainte¬ 
nance  simulators  is  increasing  gradually, 
primarilv  in  aircraft  applications  (Figure 
12). 

The  effectiveness  of  maintenance  sim¬ 
ulators  and  of  actual  equipment  trainers 
has  been  been  compared  in  about  15  studies 
conducted  from  1967  to  1980  (Figure  13); 
five  different  types  of  simulators  were 
evaluated.  The  findings  are  as  follows: 

1.  Simulators  were  as  effective  as 
actual  equipment  trainers. 

2.  Simulators  saved  some  student  time. 

3.  Students  favored  simulators  more 
than  instructors  did. 

Relatively  complete  data  on  the  acquisi 
tion  costs  of  maintenance  simulators  and  of 
comparable  actual  equipment  trainers  are 
available  in  11  cases.  Since  these  mainte¬ 
nance  simulators  are  prototype  equipments, 
they  include  costs  both  for  development  and 
for  fabricating  a  single  unit.  Therefore, 
two  estimates  were  developed  for  the  acquisi 
tion  cost  of  a  maintenance  simulator:  a  high 
estimate  that  includes  the  cost  of  research 
and  development  plus  the  cost  of  fabricating 
one  production  unit;  a  low  e  timate  that 
includes  only  the  cost  of  fabricating  one 
production  unit.  The  cost  of  an  actual 
equipment  trainer  is  the  recurring  produc¬ 
tion  cost  plus  the  cost  of  modifying  that 
unit  to  make  it  useful  at  school. 


The  acquisition  cost  of  maintenance 
simulators  is  between  0.20  to  0.60  as  much 
as  that  for  comparable  actual  equipment 
trainers,  depending  on  the  method  used  to 
estimate  the  cost  of  the  simulator  (Figure 
14).  The  lower  ratio  is  probably  the  more 
me^  Lngful  comparison  because  it  excludes 
the  costs  of  research  and  development  both 
for  simulators  and  actual  actual  equipment 
trainers.  However,  a  comparison  limited 
only  to  acquisition  costs  overlooks  the 
long  term  costs  of  using  simulators  and 
actual  equipmei.t  trainers. 

Only  one  complete  cost-effectiveness 
evaluation  of  a  maintenance  simulator 
and  its  comparable  actual  equipment  trainer 
has  been  found.  This  was  a  15  year  life- 
cycle  cost  comparison  cf  the  Air  Force 
6883  Avionics  Test  Bench  3-Dimensional 
Simulator  for  the  F-lll  aircraft  and  the 
actual  equipment  trainer  (Cicch inelli , 
Harmon,  Keller  and  Kottenstette ,  1980),. 

The  study  finds  that  both  were  equally 
effective,  as  measured  by  student  perform¬ 
ance  at  school  and  by  supervisors'  ratings 
of  performance  of  course  graduates  later 
on  the  jo*b.  Savings  due  to  use  of  the 
simulators  are  due  primarily  to  lower 
operating  costs  over  the  period  of  inter¬ 
est.  With  these  savings,  the  acquisition 
cost  of  the  simulator  can  be  amortized 
with  n  four  years  (Figure  15). 


DISCUSSION 

The  findings  of  these  studies  on  the 
cost-effectiveness  of  selected  types  of 
military  training  are  summarized  in  Figure 
16;  the  values  are  rounded  off.  The  over¬ 
all  interpretation  must  be  that  innova¬ 
tions  to  training  in  these  areas  are  as 
effective  as  the  methods  and/or  equipments 
to  which  they  were  compared.  Where  ccst 
data  are  available,  the  innovations  cGst 
l«ss  to  procure  and  use  than  the  systems 
they  can  replace.  Thus,  research  and  de¬ 
velopment  on  training  has  led  to  innova¬ 
tions  that  are  cost-effective.  /his 
overall  conclusion  must  be  qualified 
because  some  of  the  data  are  incomplete, 
as  discussed  next.  Credible  data  that 
compare  the  complete  costs  of  computer- 
based  instruction  and  conventional  in¬ 
struction  have  not  yet  been  developed. 

This  applies  also  to  the  operating  and 
life  cycle  costs  of  maintenance  simulators 
and  of  actual  equipment  trainers.  There 
is  a  need  not  only  for  more  complete  cost 
data  but  for  data  that  can  be  used  to 
identify  the  high  cost  elements  (i.e., 
cost-drivers)  in  such  training  systems; 
such  data  are  needed  also  to  develop  cost- 
estimating  relationships  for  estimating 
the  probable  costs  of  new  training  sys¬ 
tems. 

I*-  is  important  to  note  that  the 
effectiveness  of  computer-based  instruc¬ 
tion  and  of  maintenance  simulation  has 
been  demonstrated  only  on  the  basis  of 
student  achievement  at  school.  Such 
data  do  not  tell  us  how  well  these  stu¬ 
dents  perform  on  the  job,  and  job  perform- 
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FIGURE  12.  Estimated  Procurement  of  Maintenance  Trainers  (Simulators  and  Actual 

Equipment)  by  the  Department  of  Defense,  According  to  Type  of  Application, 
1977-1985  (Estimated  November  1979). 


FIGURE  13.  Studies  on  the  Effectiveness  of  Maintenance  Simulators,  1967-1980. 


FIGURE  14.  High  ind  Low  Estimates  for  tha  Acquisition  Costs  of 
Maintenance  Simulators/Actual  Equipment  Trainers. 
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FIGURE  15.  Amortization  of  the  68S3  Test  Stand.  Three-OimenJonal 

Maintenance  Simulator  (Source:  CicchinelSi.  Harmon.  Keller 
and  Kottenstette.  1980). 
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ancc  is  the  more  relevant  measure  of  train¬ 
ing  effectiveness.  This  observation 
applies  equally  to  conventional  instruc¬ 
tion.  One  exception  to  these  comments  is 
that  supervisors'  ratings  showed  about 
the  same  job  performance  for  students 
trained  either  with  the  6883  simulator  or 
actual  equipment;  however,  supervisors' 
ratings  are  not  objective  performance 
measures.  On-the-job  performance  measures 
are  an  inherent  aspect  of  evaluating  flight 
simulators  because  all  students  who  were 
trained  in  simulators  were  observed  later 
in  aircraft  in  flight,  as  were  ooviously 
those  trained  only  in  aircraft.  However, 
measures  of  student  performance  in  air¬ 
craft  are  based  on  instructors'  ratings 
on  well-developed  rating  scales  rather 
than  on  objective  measures  recorded  by 
instruments.  It  may  well  be  that,  regard- 
loss  of  method  of  training,  performance 
at  school  does  not  predict  later  perform¬ 
ance  on  the  30b.  Whether  there  is  a  high 
or  low  degree  of  relationship  between 
measures  taken  at  school  and  later  on  the 
job  is  an  important  question  that  still 
remains  to  be  answered  cor  almost  all 
types  of  training. 

A  basic  premise  of  innovations  to 
training  such  as  flight  simulators,  compu¬ 
ter-based  instruction  and  maintenance 
simulators,  is  tnat  they  will  improve  the 
quality  of  instruction  and,  thus,  the 
effectiveness  of  training.  The  basic 
findings,  as  reported  above,  are  that 
these  innovations  to  training  are  about  as 
effective  as  the  methods  they  can  replace 
and  save  appreciable  amounts  of  student 
tine.  There  have  been  only  a  few  demon¬ 
strations  of  slight  improvements,  none 
largo  enough  to  have  any  practical  signif¬ 
icance.  Overall,  this  means  that  these 
innovations  yield  improvements  in  effi¬ 
ciency,  i.e.,  cost,  but  not  in  effective¬ 
ness. 

Innovations  to  training- -based  on  what 
we  know  about  those  three  examples- -nay 
have  a  potential  for  improved  effective¬ 
ness,  but  this  has  not  been  conclusively 
demonstrated.  Superior  student  achieve¬ 
ment,  compared  to  that  with  conventional 
instruction,  was  found  in  about  one-third 
of  the  studies  of  computer-assisted  in¬ 
struction  (CAT);  the  amounts  of  siijm?  r  ior  i  fy 
were  snail.  Two  such  results  (out  of  11 
eas^s)  wore  reported  for  the  use  of 
maintenance  simulators;  none  for  flight 
simulators.  It  is  often  said  that  mainte¬ 
nance  simulators  can  provide  better  train¬ 
ing  because  they  can  demonstrate  a  larger 
number  of  naif  unctions  than  is  possible 
with  actual  equipment  trainers.  This 
plausible  claim  has  not  been  verified  nor, 
apparently,  even  tested.  Course  materials 
and  instructional  procedures  incorporated 
in  innovations  to  training  have  essentially 
been  mode  loci  on  the  current  methods.  This 
imposes  an  unnecessary  limitation  on 
improvements  in  effectiveness  that  nay 
well  be  present.  The  goal  of  almost  all 
improvements  to  traininu  so  far  examined 
scons  to  be  equal  effectiveness  and  savings 
in  the  cost  of  training;  only  lip  service 


has  been  given  to  improved  effectiveness. 

It  appears  possible  to  produce  improve¬ 
ment^  in  effectiveness  if  costs  were  kept 
constant,  i.e.,  if  large  savings  in  cost 
were  not  also  required  at  the  same  time. 

Finally,  there  has  been  no  attempt  to 
look  for  trade-offs  between  cost  and 
effectiveness.  To  do  so  would  require 
different  types  of  comparisons,  different 
experimental  designs,  and  a  greater  concern 
with  improving  training  than  is  currently 
evident.  It  would  require  control  (in 
experiments),  o'-er  the  amount  of  time 
students  spend  with  simulators  or  CAI/CMI 
to  determine  the  shape  of  the  learning 
curve  and  thus  the  optimum  time  to  go  from 
simulator  to  the  aircraft  or  actual  equip¬ 
ment. 

Additional  practice  in  training  firings 
diminishing  improvements  in  pe r formance . 

The  well-known  phenomenon  of  the  learning 
curve  and  its  impact  on  the  cost-effec¬ 
tiveness  of  training  has  been  almost 
totally  disgregarded  except  in  one  excel¬ 
lent  study.  Povenmire  and  Rosjoe  (1973) 
showed  that  the  transfer  effectiveness 
ratio,  which  measures  the  amount  of  flight 
time  saved  due  to  use  of  a  simulator,  con¬ 
tinued  to  decrease  and  eventually  became 
flat  (after  about  7  hours  in  this  particu¬ 
lar  study  where  the  task  was  for  the  pilot 
to  qualify  in  a  ''heck  flight).  When  the 
decreasing  THRs  were  compared  to  the  costs 
of  using  the  simulator  or  aircraft  in  this 
case,  it  became  clear  that  it  would  fie 
more  cost-effective  to  use  the  airplane 
rather  than  tho  simulator  for  additional 
training  after  about  4  hours  in  the  simu¬ 
lator  (and  not  after  7  or  1 1  hours,  is  was 
done  for  some  of  the  test  groups  in  this 
s tudy ) . 

A  concern  with  trade-offs  between 
cost  and  effectiveness  would  also  require 
us,  in  effect,  to  develop  simulators  that 
differ  in  complexity,  and  therefore  in 
cost,  to  see  trie  extent  to  which  effective¬ 
ness  nay  change  as  a  consequence  of  changes 
in  the  cost  of  equipment.  Up  to  the  pre¬ 
sent,  all  comparisons  have  been  made  with 
a  simulator  designed,  more  or  less  to  an 
arbitrary  level  oi  complexity,  as  the 
result  botn  of  the  Instructional  System 
Design  process  and  of  negotiations  between 
operators,  trainers,  am)  developers  over 
requirements  and  costs.  This  process 
needs  data  on  the  critical  relationship 
between  cost  and  effectiveness  in  order 
to  make  optimum  trade-offs. 

CONCLUSIONS 

Flight  simulators,  computer-based  in¬ 
struction  and  maintenance  training  simu¬ 
lators  apixiar  to  be  as  effective  as  the 
methods  of  training  they  can  replace;  they 
also  can  reduce  the  costs  of  training. 

Thus,  they  appear  to  f>e  cos  t-et  feet  i  ve 
compared  to  more  conventional  methods  of 
training.  These  new  methods  of  training 
have  not  been  demonstrated  to  bo  more 
effective  than  the  older  ones,  although  it 
is  jwssible,  if  cost  savings  were  not 
required  or  were  kept  constant,  that  they 
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might  be.  Trade-offs  between  the  effective¬ 
ness  and  cost  of  alternative  training 
systems  have  not  been  performed.  To 
do  so,  more  reliable  and  more  complete 
data  on  effectiveness  and  cost  would  be 
required  in  most  areas  of  training. 
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ABSTRACT 


Although  reimuiiiiy,  avaiiaoiiity  ana  maintainability  (K AM)  and  system  support  policies  for  Army  equipment 
in  general  are  fairly  well  defined  in  such  documents  as  AR  700-127  and  AR  702-3  or  various  MIL  STD's,  there  is 
little  that  recognizes  ur  ique  RAM  and  support  considerations  that  apply  to  training  systems.  The  authors  identify 
some  of  these  considerations,  which,  though  largely  unique  to  training  systems,  are  generic  to  most  training 
systems.  They  then  discuss  the  impact  of  these  considerations,  with  emphasis  on  how  RAM  specification  and 
growth  and  support  management  differ  from  that  of  the  combat  systems  around  whose  needs  AR  702-3  and  the 
MIL  STD's  are  principally  modelled.  The  authors  develop  specific  conclusions  as  to  policy  and  practice 
distinctions  from  the  combat  systems  model  that  should  be  made  in  training  systems  development  programs. 


INTRODUCTION 

The  subject  of  tnis  p/per  is  one  that  is  perhaps  as 
much  a  cause  of  frustration,  and  a  cost  driver,  as  any  in 
the  field  of  materiei  acquisition  management.  On  the 
face  of  it,  it  wouldn't  seem  that  it  should  be.  The 
acronym  "RAM",  Reliability,  Availability,  Maintainability, 
can  be  re-arranged  to  make  a  very  simple  mathematical 
statement 

A  =  f (R,  M) 

that  availability  of  a  system  is  a  function  of  reliability  — 
how  often  it  fails  —  and  maintainability  —  how  much  time 
it  takes  to  fix  it  and  keep  it  operating.  The 
implementation  of  this  relationship  does  not  seem  much 
more  difficult.  The  customer  presumably  knows  what 
availability  he  will  require.  Volumes  of  data  exist  on  the 
reliability  of  almost  any  class  of  components.  The 
mathematics  of  reliability  calculations  are  not  unusually 
difficult,  and  engineering  to  enhance  maintainability  is 
generally  well  dor-umentea.  Go,  is  it  really  necessary  for 
this  almost  trivially  simple  seeming  relationship  to  cause 
so  much  heartache  and  cost  so  much  money? 

The  answer,  of  course,  is  that  it  is  necessary  only  to 
the  extent  that  we  accept  Murphy  s  Law  as  an  inviolable 
rule  of  nature.  In  our  more  rational  moments,  as  when  wc 
are  contemplating  someone  elses’  problems,  we  know 
Muiphy's  Law  to  be  mostly  a  consequence  of  human 
carelessness  and  poor  judgement.  We  ore,  then,  led  to  a 
conclusion  that  most  RAM  fiascos,  as  tnos  other 
manifestations  of  Murphy’s  Law,  can  be  avoided  by  using 
more  care  and  better  judgement. 

Recognizing  that  :^at  last  piece  of  advice,  left 
standing  alone,  is  not  particularly  useful,  the  Army  has 
created  a  substantial  body  of  regulations,  standards  and 
methodology  to  guide  our  judgement  and  prompt  our 
memories.  This  has  evolved  over  many  years  and,  in  our 
opinion,  when  applied  to  types  of  systems  wiu)  which  the 
Army  has  had  extensive  experience,  represents  an 
excellent  prescription  for  achieving  the  required  level  of 
reliability,  availability  and  maintainability  within  a  stated 
cost  and  time.  Indeed,  we  would  suggest  that  most 
failures  of  RAM  programs  in  conventional  military 
hardware  development  have  occurred  when  the  developer* 


put  their  RAM  engineering  emphasis  in  areas  with 
minimum  payback  and  tried  to  take  shortcuts  in  some  of 
the  important  ones.  As  the  hobbit,  Bilbo  Baggins, 
observed  "Shortcuts  make  for  long  delays".  This  is 
particularly  true  if  the  short  cuts  are  in  the  wrong  areas. 

The  previous  observation  applies,  of  course,  to 
systems  where  the  paths  have  been  trod  before  and  the 
way  has  become  well  charted,.  Even  with  those  systems, 
there  was  a  period  when  delay  and  expense  were  incurred 
not  due  to  any  shortcutting,  but  just  from  being  down  on 
the  learning  curve.  As  we  came  up  on  the  learning  curve, 
the  guidance  could  be  formulated  and  then  extended  to 
other  type  systems,  where,  again  ’here  would  be  a 
learnirg  period  until  the  variations  in  the  guidance 
applicable  to  the  new  type  systems  could  be  recognized. 

The  growth  in  recent  years  of  complex  major 
training  simulation  systems  has  presented  us  with  the 
maturing  of  just  such  a  set  of  altered  RAM  guidelines. 
We,  the  authors,  have  worked  for  the  past  several  years  in 
RAM  development  and  management  of  these  training 
devices  within  the  general  context  of  AR  702-3,  the 
primary  source  for  RAM  development  guidance  in  Army 
materiel  acquisition  programs.  As  with  any  good 
regulation,  intelligent  interpretations,  weighing  payoffs 
against  cost  and  time,  are  possible,  and  we  have  made 
them,  and  seen  them  made,  some  more  successfully  than 
others.  With  ou-  snare  of  bruises  from  climbing  up  the 
learning  curve,  we  feel  we  can  now  speak  with  some 
authority  on  what  these  emerging  guidelines  are.  The 
interesting  aspect  of  this  is  that,  as  yet,  the  rest  of  the 
user  and  development  community,  to  include  RAM  people, 
whe  have  not  yet  had  this  experience  naturally  think  in 
terms  of  the  standard  procedures  and  each  new  project 
and  each  new  person  coming  ir.to  the  project  represents  a 
new  education  effort.  This  conference  is,  therefore,  a 
timely  opportunity  to  get  some  of  our  ideas  and 
experience  before  the  training  device  community,  discuss 
them,  and  suggest  some  steps  for  recognition  of  changes 
in  RAM  policy  and  procedure  that  will  expedite 
development  of  adequately  available  and  supportable 
trainir^  simulation  system,-. 
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RAM  REQUIREMENT  DEFINITION 

As  we  mentioned  earlier,  RAM  performance  is 
something  that  costs  money  and  takes  time.  The  users' 
bottom  line  is  being  able  to  count  on  the  system  when  he 
needs  it,  for  as  long  as  he  needs  it.  In  saying  this,  it  is 
important  to  understand  what  overstated  RAM  require¬ 
ments  mean  to  the  developer  and  the  logistician.  The 
developer  is  com  nitted  to  high  reliabi’ity  parts,  heavily 
derated  circuits,  rigorous  QA  inspections  and  redundancy 
ip  mission  essential  functions.  When  the  thing  goes  down, 
we  can't  take  the  time  to  isolate  the  failure  to  the  piece 
part,  nor  to  send  for  the  technician  and  equipment  that 
could,  so  we  will  replace  the  obviously  failed  major 
assembly  and  be  back  up  in  a  few  minutes.  That  the 
logistician  is  now  stocking  the  supply  system  with  these 
major  assemblies,  instead  of  just  replacements  for  the 
failed  parts,  is  something  we  have  to  decide  to  afford. 
The  off-line  time  to  repair  the  major  assemblies  has 
become  a  secondary  consideration,  way  behind  that  on¬ 
line  tinu  to  get  the  system  back  up. 

With  combat  systems,  when  failure  or  unavailability 
can  get  people  killed  and  battles  lost,  it  is  possible  to 
justify  some  very  expensive  RAM  require '..ents.  In  the 
Concept  Formulation  Phase  of  the  Life  Cycle 
Management  Model  (DA  PM  11-25)  a  generic  process  for 
all  Army  materiel  acquisition  is  spe.led  out  for 
justification  of  requirements,  to  include  RAM 
requirements.  For  this  process  to  work  properly,  it  must 
be  carried  out  jointly  by  the  user  and  the  developer  in 
three  distinct  and  separate  steps.  First  the  user  must 
determine,  in  profiles  of  all  missions  and  a  .•.ummary  of  all 
operational  modes,  just  how  the  system  will  be  used 
throughout  its  life.  This  Operational  Mode 
Summary/Mission  Profile  document  can  then  be  used  by 
the  user  and  developer  to  jointly  determine  a  reali  tie 
definition  of  just  what  will  constitute  a  mission  or  system 
failure  and  define  objective  criteria  for  determining  how 
incidents  are  to  be  classified.  With  the  Operational 
Modes  summary.  Mission  Profile  (OMS  MP),  and  Failure 
Definition  and  Scoring  Criteria  (FI)  SC)  in  hand,  the  user 
and  developer  can  :*en  jointly  develop  numerical  RAM 
requirements  that  realistically  relate  operational 
readiness  and  mission  success  to  acquisition  cost  and 
logistic  supportability.  The  basis  for  these  requirements 
is  stated  in  a  RAM  Rationale  Annex  to  the  official 
requirements  document  used  to  establish  the  development 
program.  The  methodology  for  preparing  this  annex  and 
therefore  for  arriving  at  the  numerical  values,  is 
excellently  stated  in  ihe  joint  TR ADOC/DARCOM  RAM 
Rationale  Annex  Handbook.  Besides  calling  for  the 
OMS/MP  and  FD/SC  as  input,  this  procedure  also  requires 
consideration  of  the  R  AM  assumptions  made  in  the  user’s 
Cost  and  Operational 'Training  Effectivenes.  Analysis 
(COEA/CTKA)  and  of  the  developer's  best  techn'cal 
approach  analysis  performed  in  Concept  Formulation. 
The  user  then  performs  an  analysis  to  determine  the 
minimum  acceptable  values  that  will  permit  the  system  to 
be  useful  to  him.  The  developer  determines  the  best 
operational  capabilities  that  are  irv*  rrent  to  the  selected 
technical  approach  and  attainable  within  the  projected 
development  program.  Hopefully,  these  will  differ  by 
enough  and  in  the  prop*.  direction,  to  permit  initiation  of 
a  program  with  realistic  cost  and  mission  effective  RAM 
program  objectives. 

It  should  be  noted,  in  regard  to  reliability 
qualification  testing,  ttal  recent  changes  to  AR  702  -3  and 
the  joint  OTE  \/TR>  DOC/DARCOM  baseline  FD/SC 
extend  operational  failure  chargabil.  beyond  contractor 
produced  hardware  to  ine*  ide  Government  furnished 


equipment  and  the  total  support  system.  This  will 
significantly  increase  the  number  of  items  which  can 
result  in  an  unsatisfactory  RAM  report  card.  Training 
Device  contractors  should  be  aware  of  this  important  new 
change  and  thrust  in  the  RAM  area. 

We  said  that  we  were  -oing  to  discuss  variations 
from  the  standard  RAM  procedures  that  we  considered 
applicable  to  training  devices,  but,  in  this  case,  we  find 
the  generic  model  to  be  admirably  applicable  to  training 
devices  as  it  stands.  The  unique  feature  in  this  case  is 
that  the  generic  model  does  not  get  applied  to  training 
device  requirements  definition  as  often  or  as  thoroughly 
as  it  should.  The  RAM  requirements  analysis  is  usually 
done  quite  thoroughly  for  major  combat  systems.  After 
all.  the  Army  cannot  afford  to  either  gold-plate  or  come 
up  a  loser  on  such  highly  visible  projects.  The  analysis  is 
usually  performed,  albeit  sometimes  somewhat 
perfunctorily,  (e.g.,  rarely  is  a  CTEA  available  in  time  to 
support  preparation  of  the  RAM  Rationale  Annex)  on 
lesser  systems,  to  include  stand-alone  or  non-system 
derivative  training  devices.  The  problem  really  arises 
when  the  training  package  is  included  in  the  overall 
development  program  of  a  major  system.  Priority  goes  to 
definition  of  the  RAM  requirements  of  the  major  system 
itself  and  its  key  support  equipment.  There  doesn’t 
usually  seem  to  be  enough  left  over  to  get  around  to  the 
training  devices,  at  least  in  time  for  the  statement  of 
requirements  to  have  any  effect  on  the  development 
program.  What  results  is  that  the  users  wind  up  shaping 
their  program  around  what  they  are  going  to  get,  instead 
of  telling  the  developers  what  they  need  in  order  to  run 
the  program  they  want. 

RELIABILITY  PROGRAM  TASKS 

An  essential  reference  in  preparing  or  evaluating 
any  reliability  program  is  MIL-STD-785B.  Th's  contains 
descriptions  and  guidance  on  the  use  of  the  various 
reliability  engineering,  management  and  accounting  tasks 
that  may  comprise  the  reliability  program  of  any  material 
acquisition  program.  This  M1L-STD  enables  the  developer 
to  put  together  a  reliability  program  optimized  around  the 
needs  of  his  particular  project,  to  include  training 
devices.  These  tasks,  and  their  recommended 
applicability  to  the  major  phases  of  a  material  acquisition 
program,  are  shown  in  Figure  1. 

While  these  tasks  are  written  with  the  objective  of 
general  applicability,  there  are  a  nu.nbcr  of 
considerations  that  guide  their  selection  and  use  in 
training  device  programs.  Many  of  these  will  be  touched 
on  later  in  this  paper.  One  aspect  of  training  device 
procurement,  however,  cuts  across  a  number  of  these 
tasks  and  should  be  pointed  out  in  this  discussion.  This  is 
tl.e  tendancy  of  a  training  device  developer  to  use  more 
commercially  available  major  assemblies  and  less 
development  from  the  piece  part  level  of  major 
assemblies  unique  to  the  system.  Thus  monitoring  and 
controlling  vendor  supplied  items  is  keyed  more  to 
performance  of  fewer,  more  complex  items  and  less  to 
inspee’ion  of  a  large  number  of  smeller  parts.  Failure 
Reporting  Analysis  and  Corrective  Action  Systems 
(FRACAS)  and  Failure  Review  Boards  must  deal  with 
failures  within  these  vendor  assemblies  that  are  not 
necessarily  controllable  by  the  developer.  The  analytical 
process.  Failure  Modes  Effects  and  Criticality  Analysis 
(FMEOA)  and  reliability  modeling,  allocation  and 
prediction  are  facilitated,  but  also  constrained  by  the 
predetermined  characteristics  of  these  fewer,  larger 
assemblies.  Reliability  growth,  which,  as  we  shall  see, 
needs  help,  can  get  it  by  taking  advantage  of  *.he 
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APPLICATION  MATRIX 


MIL-STD-785B 
APPENDIX  A 

MATRIX  15  8EPTEMBER  1080 

TASK.  PROGRAM  PHASE _ 

TYPE  CNCPT  I  VALID  I  FSED  I  PROD 


S ( 2 )  G(2)  G ( 2 ) 


(1)(2)  ( 1 )  ( 2 ) 

G(l)  GC(1) 


S(2)( 3)  G ( 2 ) 
S(l)  G 

S(l)  .G 


G(2) 

G( 2 ) { 3 ) 


TASK  TYPE: 

ACC  -  RELIABILITY  ACCOUNTING 
CNQ  -  RELIABILITY  ENGINEERING 
MOT  -  MANAGEMENT 


PROjg^AMjPHm; 

S  -  SELECTIVELY  APPLICABLE 
G  -  GENERALLY  APPLICABLE 

GC  -  GENERALLY  APPLICABLE  TO  DESIGN  CHANGES 
ONLY 

NA  -  NOT  APPLICABLE 

It)  -  REQUIRES  CONSIDERABLE  INTERPRETATION 
OF  INTENT  TO  BE  COST  EFFECTIVE 
U>  -  MIL-STD-733  IS  NOT  THE  PRIMARY 
IMPLEMENTATION  REQUIREMENT. 

OTHER  MtL-STDS  OR  STATEMENT  OF 
WORK  REQUIREMENTS  MUST  BE 
INCLUDED  TO  DEFINE  TH  {  REQUIREMENTS. 

FIGURE  1 
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demonstrated  reliability  within  proven  components.  In 
summary,  the  reliability  program  for  a  training  device 
must  consider  and  take  advantage  of  the  reliability 
program  previously  earned  out  in  major  developed 
assemblies  which  the  training  device  developer  has 
generally  greater  latitude  to  use. 

AVAILABILITY 

An  obvious  difference  between  combat  systems  and 
training  devices  is  that  in  controlling  and  scheduling  the 
use  of  a  combat  system,  one  must  concede  some  degree  of 
initiative  to  the  enemy  and  provide  a  great  deal  of 
flexibility  to  respond  to  unexpected  requirements.  A 
training  manager,  on  the  other  hand,  generally  has  to  have 
complete  control  of  his  scheduling  and  rarely  does  he  have 
to  schedule  equipment  around  the  clock.  Thus  the  typical 
availability  requirement  for  a  combat  system  is  long 
periods  of  standby  in  readiness  for  immediate  response  in 
periods  of  short  intense  use.  This  leads  to  definition  of 
operational  availability  in  AR  702-3  in  terms  of  total 
available  tim*.  within  a  stated  calendar  period.  Any 
particular  calendar  period  of  use  of  a  training  system, 
however,  is  going  to  include  scheduled  periods  of 
downtime,  i.e.,  periods  when  the  system  is  not  required  to 
be  available.  Describing  operational  availability  as  in  AR 
702-3  results,  therefore,  in  stating  an  availability 
requirement  that  does  not,  in  fact,  exist.  PM  TRADE  and 
the  Naval  Training  Equipment  Center  have  developed  and 
used  for  a  number  of  years  availability  characteristics 
based  on  schedulled  usage.  An  article  in  the  Army 
Logistic  Center's  RAM/ILS  Bulletin  of  November  1911 
addresses  this  case  and  provides  an  operational 
availability  definition  that  uses  only  the  system's 
availability  during  the  periods  when  it  is  recuired  to  be 
available.  Publication  in  this  bulletin  has  not  had  a 
significant  impact  on  training  device  availability 
specification;  however,  we  anticipate  bringing  tne  idea  to 
the  forefront  in  future  documents  in  order  that  it  will 
take  a  more  meaningful  place  in  training  device 
availability  considerations. 

RISKS 

Risks  are  the  considerations  which  cause  ./otne 
failure  modes  to  be  more  critical  than  others  and 
^liability  of  some  components  to  be  more  critical  than 
others.  In  combat  systems,  these  risks  are  generally  those 
that  lead  to  mission  failure.  Preventing  mission  failure  is 
also,  of  course,  a  major  objective  of  reliability 
engineering  of  training  devices,  too.  We  have,  however, 
already  observed  that  the  generally  less  cetastrophic 
consequences  of  failure  of  a  training  mission  affect  the 
level  of  reliability  that  we  can  justify  paying  for.  There 
are,  however,  some  risks  that  require  more  stringent 
control  than  docs  the  combat  system.  No  one  wants  a 
combat  system  to  be  unsafe  for  its  crew,  but  the  level  of 
safety  that  k  sought  reflects  the  fact  that  the  battlefield 
is  an  inherently  dangerous  place.  By  contrast,  death  or 
injury  in  the  training  environment  are  unacceptable. 
Combat  is  also  rather  hard  on  the  natural  environment, 
while  preservation  of  the  environment  is  always  a  major 
concern  of  the  training  manager.  These  two 
considerations  frequently  iihibit  the  use  of  combat 
systems  in  training  and  make  training  simulation 
necessary.  For  example,  one  of  the  reasons  for  turret 
maintenance  trainers  is  to  permit  students  to  work  on  tne 
drive,  hydraulic  ard  electrical  systems  without  being 
expaacd  to  the  letnal  levels  of  force,  pressure  and 
voltage  that  are  present  in  the  actual  turret.  Therefore, 
while  one  can  perhaps  settle  for  a  lesser  probability  of 
mission  SMecem  in  a  training  device,  reliability 


engineering  must  do  its  part  to  absolutely  preclude  injury 
or  environmental  damage,  risks  which  are  generally 
acceptable  at  some  level  in  combat  systems.  A  good 
FMECA  a.,d  a  FRACAS  program  are  therefore  no  less 
necessary  in  the  development  of  a  training  device.  The 
determinants  of  criticality  may,  however,  differ  from  like 
analyses  and  programs  in  developing  a  combat  system. 

ENVIRONMENTAL  DESIGN  CRITERIA 

The  combat  system  must  work  in  whatever 
environment  the  combat  takes  place.  Thus  AR  70-38  and 
MIL -STD-8 10  prescribe  some  demanding  and  expensive 
environmental  design  criteria:  heat,  cold,  humidity,  dust, 
moisture,  shock  vibration,  etc.  The  training  system,  of 
course,  must  work  where  the  training  takes  place  and,  fer 
some  systems,  such  as  the  Multiple  Integrated  Laser 
Engagement  System  (MILES),  that  is  essentially  the  same 
as  the  combat  system  being  simulated.  Many  training 
simulators  can,  however,  be  used  in  a  fully  controlled 
environmental  shelter,  subject  to  movement  only  under 
administrative  conditions  and  generally  can  live  with  the 
same  environmental  design  criteria  as  arc  commercially 
applied  to  fixed,  interior  electronic  equipment.  Even 
those  training  devices  that  are  used  in  the  same 
environment  as  the  combat  systems  can  enjoy  some 
measures  of  relief  from  at  least  the  extreme 
environmental  criteria  imposed  on  the  combat  system.  As 
we  have  previously  observed,  the  training  manager  has 
control  over  when  his  equipment  will  ot  used  in  extreme 
climatic  conditions,  and  also,  only  a  relatively  small 
proportion  of  training  occurs  under  those  conditions.  This 
is  in  contrast  with  most  combat  systems,  where  any  one 
item  must  be  inherently  capable  of  quick  deployment  and 
use  to  counter  any  threat.  This  suggests  that  the  training 
device  can  be  designed  to  criteria  that  are  less  demanding 
than  those  of  the  combat  system,  and  a  kit  or 
modification  procedure  provided  for  those  relatively 
infrequent  and  scheduled  periods  of  use  in  more  seve-e 
conditions.  Thus,  while  the  combat  systems  on  whi«v. 
MILES  is  used  were  designed  for  use  in  the  full  range  of 
the  former  AR  70-38  climatic  categories,  MILES  was 
specified  to  only  the  intermediate  categories  5  and  6. 

DURABILITY 

The  preceding  discussion  suggested  a  possible 
relaxing  of  standards  for  the  severity  of  the  use 
environment.  Paradoxically,  this  paragraph  is  going  to 
suggest  that  in  the  area  of  durability  i.e.,  resistance  to 
wearing  out,  what  is  adequate  for  the  combat  system  may 
not  be  adequa  -  for  the  training  device.  Consider  that 
the  gunnery  covrols  of  a  tank  ere  used  only  when  that 
crew  uses  that  tank  m  gunnery  training  «.<r  an  engagement. 
It  is  very  important  that  the  controls  work,  but  it 
probably  isn't  subject  to  very  many  use  hours  per  year. 
Now  consider  the  same  controls  in  the  Armor  Unit 
Conduct-of-Fir*  Trainer.  These  ’/ill  be  used  all  day, 
every  day  by  every  crew  in  tbt  battalion,  probably  more 
hours  in  a  year  than  most  tank  controls  will  see  in  the  life 
of  the  tank.  Or  consider  f»n  installation  set  of  MILES 
equipment.  Each  unit  scheduled  to  use  it  will  come  out, 
put  it  on  their  weapons,  run  the  exercises,  take  the  MILES 
gear  off  and,  after  restoring  the  weapons  to  inspection 
condition,  replace  them  in  the  arms  room.  The  MILES 
gear,  on  the  other  hand,  will  be  inventoried,  maybe  get  a 
quick  check  for  serviceability,  and  be  back  out  in  the  field 
with  another  unit  the  next  week.  Clearly  use  and  wear 
out  factors  that  determine  the  replacement  and  rebuild 
periods  in  the  combat  system  must  be  reconsidered  when 
the  same  component  is  used  in  the  training  device. 
Training  systems  that  are  of  comparable  durability  and 
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used  on  an  hour-for-hour  basis  with  combs;  system  may  in 
fact  see  a  much  shorter  calendar  life  between 
replacement  or  rebuild.  These  are  considerations  that 
need  not  cause  any  problems  if  they  are  appreciated  and 
dealt  with  in  the  life  cycle  support  planning  for  the 
system,  but  they  do  cause  some  nasty  surprises 
downstream  if  they  are  overlooked. 

RAM  GROWTH  RATE 

RAM  growth  management  and  projection  are  a 
normal  part  of  any  materiel  development  program  and  the 
methodology  for  doing  this  in  training  devices  is  not 
different  from  that  of  other  comparable  materiel. 
Several  inherent  characteristics  of  the  development 
schedules  for  training  devices  do,  however,  have  an 
interesting  impact  on  training  device  RAM  growth.  First 
of  all,  design  of  a  training  device  to  simulate  a  combat 
system  must  unavoidably  lag  the  design  of  the  system. 
Changes  after  the  system  design  is  supposed  to  be  firm 
will  play  crack-the-whip  with  the  training  device  design. 
Thus  the  design  effort  on  which  the  RAM  growth  is  based 
starts  later  and  suffers  more  rapid  perturbation  than  does 
the  design  of  the  system  itself.  At  the  other  end  of  the 
schedule  lies  operational  test  and  evaluation  of  the 
system,  which  at  least  theoretically  includes  use  of  the 
operational  training  package  to  train  test  personnel. 
Evaluation  of  that  package,  to  include  devices,  is  one  of 
the  objectives  of  the  OT<ScE.  Inus  the  training  device 
developmental  period,  in  which  the  Army’s  Life  Cycle 
Management  Model  concentrates  most  of  the  RAM  growth 
of  a  system,  is  inherently  truncated  at  both  ends,  as 
compared  to  that  of  the  combat  system,  and  perturbed  in 
the  middle. 

Figure  2  shows  the  effect  of  this.  In  order  to  have  a 
training  device  that  can  support  the  training  package  the 
device  must  be  available,  at  its  full  scale  development 
(FSD)  reliability  level,  before  the  FSD  prototype  of  the 
combat  system,  yet  design  cannot  start  until  the  combat 
system  design  is  firm.  Thus  it  has  an  inherently  steeper 
growth  rat®  than  the  combat  system  (and  those  are 
notoriously  not  conservative),  yet  it  is  a  lower  priority 
system.  Not  surprisingly,  very  few,  if  any,  combat 
systems  related  training  devices  have  ever  met  the 
schedule  of  their  major  combat  system.  What  usually 
happens  is  shown  by  the  dotted  line,  where  the  attainable 
growth  rate  causes  the  training  device  to  reach  the 
desired  RAM  goals  at  a  later  time  than  the  combat 
system  does.  If  the  training  device  design  can  perform 
the  necessary  funct'^ns  at  the  start  of  OT  and  pre¬ 
deployment  training  this  lag  is  not  necessarily 
catastrophic,  provided  some  realism  has  been  employed  in 
planning  the  initial  training  cycles.  Specifically,  that 
means  that  the  attainable  reliability  in  the  prototypes 
that  will  be  available  in  the  early  training  cycle  must  r>e 
recognized  and  compensated  for.  F°r  example,  more 
intensive  contractor  surport,  rather  than  the  test  system 
support  package,  could  be  used  to  provide  the  required 
availability.  This  would  also  apply  during  the  early  cycles 
cf  post  deployment  training.  The  test  program  should  be 
jet  up  to  provide  realistic  interim  RAM  goals  during  the 
formal  test  period,  with  follow  on  RAM  growth  and 
testing  continuing  until  the  fully  matured  RAM  goals  are 
met  on  an  attainable  schedule.  The  cost  of  more 
intensive  contractor  support  or  other  measures  to  boost 
availability  during  the  RAM  maturation  phase  is  not  really 
an  added  expense;  it  is  a  recognition,  before  rather  than 
after  the  fact,  of  a  cost  that  is  inherently  there  when 
piocurement  of  the  trainii  ^  simulator  is  tied  to  both  the 
design  and  the  schedule  of  a  major  combat  system. 
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TRAINING  DEVICE  RELIABILITY  GROWTH 

PNKMI  t 

SUPPORT  CONCEPT 

't  is  in  the  area  of  m  support  that  some  of  the 
most  significant  differences  between  combat  and  training 
system  use  environments  exists.  Fundamental  objectives 
of  combat  system  support  ere  to  keep  maintenance  and 
repair  of  an  item  as  far  forward  as  possible  and  to 
minimize  the  skills  und  special  equipment  required 
forward  to  do  it.  A  great  deal  of  ingenuity  has  gone  into 
meeting  these  two  seemingly  countervailing  requirements, 
and  the  results,  as  noted  m  the  introduction,  are  not 
inexpensive.  The  support  system  must  also  be  fully 
transportable  with  the  combat  system,  to  include 
operating  under  combat  conditions.  An  effect  of  this  is 
that  the  support  must  be  provided  by  uniformed  personnel 
and  the  allowable  tasks  become  constrained  by  the  skills 
that  can  be  taught  in  the  military  training  context.  It 
also  means  that  support  equipment  must  be  transportable 
and  operable  under  some  very  adverse  conditions.  It 
means  that  everything  needed  to  operate  and  maintain  the 
system  must  be  available  through  the  Army  supply 
system.  In  summary,  all  the  way  back  to  its  roots,  the 
support  system  is,  or  can  be,  detained  from  civilian 
industrial  or  commercial  resources. 

Obviously,  such  extreme  support  measures  apply  to 
few  if  any  training  devices.  Those  tha»  arc  placed  in  the 
TO&E  of  a  tactical  unit  come  closest  to  such  support 
requirements.  Such  devices  would  require  die  totally 
transportable  and  Army  contained  support  system.  Since 
the  training  systems  will  inevitably  have  a  lower  priority 
than  the  unit's  weapons*,  transportation  and 
communications  systems,  it  is  most  important  that  suen  a 
training  dev>e  impose  a  mL.imum  burden  on  the  owning 
unit.  This  means  no  additional  skills  can  be  required.  It 
means  minimum  maintenance,  parts  stockage  or 
additional  tools  and  test  equipment.  These  conditions 
have  generally  limited  TOAE  training  equipment  to  very 
simple  devices. 
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The  more  common  case,  applicable  to  more  complex 
devices  in  which  the  support  requirements  become  more 
demanding,  are  *hose  where  the  device  is  treated  as  part 
of  the  Post,  Camp  and  Station  property.  It  may,  in  fact, 
be  on  fhe  property  book  of  a  tactical  unit,  but  not  as  an 
item  to  be  carried  to  the  field  and  through  combat  with 
them.  If  the  unit  moves  to  another  station,  the  trainer 
would  be  shipped  administratively.  Obviously,  a  lot  more 
options  are  available  for  supporting  a  training  system  in 
this  environment  than  in  the  full  TO&F.  environment.  It 
remains  true  that  use  and  support  of  the  device  should  not 
be  a  burden  to  the  unit.  Yet  we  are  now  dealing  with 
devices  that  require  some  unique  skills,  training  and 
logistic  effort  to  use  and  support.  We  must  obviously  take 
advantage  of  the  access  to  fixed  facilities  and  civilian 
industrial  skills  that  the  garrison  environment  affords. 
Operator ''organizational  maintenance  becomes  reduced  to 
simple  GO-NO  GO  checks  with  turn-in  to  a  central 
facility  (or  facility  contact  team)  for  any  failed  units. 
The  trainee  operator  is  not  to  be  burdened  with  any 
additional  learning  in  order  to  be  able  to  use  his  training 
device.  The  central  facility  will  therefore  provide  any 
unique  operating  skills  that  may  be  needed.  This  centrax 
facility,  which  can  be  either  a  contract  operation  or  a 
Government  industrial  one,  also  equates  to  the  DS/GS 
level  in  the  combat  system  support  model.  It  should, 
however,  be  able  to  effect  significant  economies  over  a 
leld  DS/GS  operation  due  to  its  personnel  stability  and 
technical  skills,  access  to  fixed  industrial  facilities  and 
commercial  sources  and  the  scheduled  nature  of  training 
equipment  use. 


TYPE  CLASSIFICATION 

The  environment  in  which  the  training  device  is  uLed 
in  a  scheduled  manner,  at  a  given  station  and  supported 
out  of  an  industrial  facility  is,  we  have  seen,  a  major 
departure  from  the  one  in  which  most  type  classified 
standard  Army  systems  are  used.  The  check  list  of  plans, 
studies,  tests,  reports  and  evaluations  by  which  the 
materiel  acqui  ition  decision  process  arrives  at  a  type 
classification  standard  decision  is  necessary  to  insure  that 
ail  elements  of  that  transportable  self-contained  support 
system  are  in  place  and  balanced  with  regard  to  each 
other.  It  goes  far  beyond  merely  ensuring  that  the 
performance  of  the  system  is  adequate.  It  insures  that  all 
parts,  tools,  and  test  equipment  are  correctly  entered  in 
the  Army  supply  system,  all  skills  necessary  to  use  and 
support  the  systeu  are  correctly  identified,  that  stand¬ 
alone  literature  is  in  place,  that  everything  is 
quantitatively  distributed  where  and  when  it  will  be 
needed  and  can  be  moved  as  tactical  or  stra’egic 
exigencies  may  dictate. 

No  one  would  say  that  e  garrison  use  training  system 
should  be  put  in  place  without  adequate  planning  for  its 
support,  but  some  economies  relative  to  the  effort  for  a 
worldwide  combat  survivable  support  system  may  be 
attainable.  Let  us  look  at  what  is  really  required.  The 
Government  should  require  the  contractor  to  develop  and 
document  a  complete  technical  description,  identify, 
describe  and  validate  ail  maintenance  and  repair  tasks, 
along  with  the  skills,  tools  and  written  instructions  to 
perform  them,  establish  parts  stockage  and  ,se  rates  and 
validate  commercial  transportability.  These  are  the 
thing's  the  Government  needs  to  set  up  it?  own  industrial 
support  operation  or  to  "should  cost"  or  compete  a 
contractor  support  operation.  They  are  still  a  far  cry 
front  MIL  STD  documentation,  accession  of  all  end  items, 
parts  and  tool  to  the  Army  supply  system,  Arn.y  peisonnel 
and  force  structure  realignments  and  school  curricul  im 


changes.  Appreciable  savings  in  cost  and  time  have  been 
realized  when  acceptance  procedures  for  such  training 
devices  have  recognized  the  differences  in  support 
environment  from  a  standard  combat  system. 

SUMMARY 

We  have  seen  that  the  Army's  policies  and  procedure 
for  defining  and  specifying  RAM  requirements  and 
managing  the  attainment  of  those  requirements  are 
largely  modelled  around  combat  systems,  though  they 
contain  the  flexibility  to  adjust  to  other  systems.  The 
training  system  sometimes  lacks  priority,  in  competition 
with  the  combat  system,  to  get  a  timely  and  adequate 
definition  of  its  RAM  requirements.  There  are  also  a 
number  of  differences  in  the  use  environment, 
development  processes  and  support  concept  of  training 
devices  that  require  the  application  of  that  flexibility. 
Component  and  part  vs.  major  assembly  considerations 
effect  the  reliability  program.  Availability  definitions 
need  to  take  full  advantage  of  the  training  managers 
greater  control  over  how  and  when  the  device  will  be  used 
and  of  regularly  scheduled  periods  of  downtime.  Risks  of 
mission  failure  that  are  unacceptable  in  a  combat  system 
must  be  balanced  against  cost  in  a  training  system,  but 
risks  of  injury  and  environmental  damage  that  can  be 
traded  for  mission  performance  in  a  combat  system 
become  unacceptable  in  a  training  system.  We  can  take 
advantage  of  the  generally  more  benign  and  controllable 
environment  in  which  training  devices  are  used  to  relax 
some  of  the  very  severe  environmental  standards  to  wh.ch 
combat  systems  are  built.  On  the  other  hand  the  more 
frequent  use  that  training  devices  receive  means  that  we 
must  either  build  more  durability  into  them  or  accept 
mere  frequent  rebuild  or  replacement.  We  cannot  expect 
the  training  device  to  achieve  RAM  maturity  in  the  same 
growth  pattern,  relative  to  the  decision  points,  that  the 
combat  system  does,  if  we  must  inherently  curtail  and 
perturb  its  design  and  development  period  and  assign  it  a 
lower  priority.  The  training  device  has  access  to  a  much 
less  expensive  and  laborious  support  concept  than  is 
necessary  for  combat  systems.  This,  in  turn,  means  that 
some  (by  no  means,  all)  uf  the  steps  in  justifying  type 
classification  standard  of  a  combat  system  are  not 
necessary  for  training  systems. 
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LOW  COST  AIRCREW  TRAINING  SYSTEMS 
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4- -  \  ABSTRACT 

-^Two  prototype  low-cost  systems  have  been  developed  for  air  crew  training.  These  systems  provide 
instruction  in  cockpit  procedures  and  various  flight  tasks  at  approximately  one  quarter  the  cost  of 
conventional  training  approaches.  Savings  are  estimated  to  be  $1.5  million  for  one  o(  the  low-cost 
systems,  a  cockpit  procedures  trainer  for  the  SH-3H  aircraft;  $3.2  million  savings  are  estimated  for 
the  other  low-cost  system,  a  part  task  trainer  for  the  EA-3B  aircraft.  This  report  discusses  the 
cost-savir.g  approaches  and  the  acceptability  and  cost  effectiveness,  associated  with  these  two 
developments.  Efforts  to  translate  the  low-cost  approaches  to  several  ^’follow-on**  production  systems 
are  discussed.  Research  and  development  plans  for  further  improving  low-cost  training  technologies 


also  are  described 


INTRODUCTION 

The  military  requires  training  systems 
that  cost  less  than  current  systems  and  still 
perform  at  least  as  well.  Costs  to  be  saved 
include  not  just  dollars,  but  also  personnel, 
energy  resources  and  time  -  time  for  system 
development,  maintenance  efforts  and  oper¬ 
ator/instructor  personnel. 

Unnecessarily  high  training  costs  in  any 
of  these  resource  areas  always  have  been 
undesirable.  Nevertheless,  for  the  most  part, 
the  training  community  received  the  resources 
they  requested  and  used  the  resources  to 
provide  useful  but  excessively  expensive 
training  systems.  High  training  costs  are  no 
longer  just  undesirable;  they  are  intoler¬ 
able.  No  longer  *re  ample  resources  available 
for  training.  Tf  "ilitary  training  is  not  as 
eff icier  t  as  it  can  be,  there  will  not  be 
enough  resources  to  go  around,  and  the  result 
will  be  reduced  Navy  effectiveness.  Research 
and  development  (kaD)  offers  a  possible  solu¬ 
tion  to  this  problem  by  showing  how  to  build 
more  cost  effective  training  systems.  Such 
R&D  should  allow  wider  distribution  of  effec¬ 
tive  training  systems  in  the  Fleet,  with 
consequent  benefits  to  Naval  operations. 

In  pursuit  of  this  R&D  solution,  the  ulti¬ 
mate  goal  of  the  P4D  program  discussed  in  this 
report  la  to  improve  the  process  for  acquiring 
"low-cost"  training  systems,  i.e.,  systems 
that  are  lower  in  cost  (in  all  critical 
resource  ar*»as)  than  conventional  systems,  but 
no  less  effective.  To  achieve  this  goal, 
low-cost  training  systems  are  conceived, 
designed  and  developed,  and  then  implemented 
and  evaluated  in  operational  settings. 
Efforts  are  made  not  only  to  describe  the  cost 
savin?  features  of  the  R&D  developments,  but 
also  to  formulate  general  procedures  and 
rationale  for  the  de  ;ign  of  additional  and 
better  low-cost  systems. 

Distinguishing  features  of  this  R&D  are: 

•  Comprehensiveness  -  To  assure  that 
resources  saved  at  one  point  are  not 


paid  back  at  another,  this  R&D  is 
concerned  with  all  phases  in  the  life 
cycle  of  a  training  system,  from 
conception  through  obsolescence. 

•  Operational  Implementation  -  Heavy 
emphasis  is  given  to  urgent  opera¬ 
tional  requirements  for  use  of  the 
products  from  these  projects. 

•  Eclectic  Approaches  -  A  variety  of 
diverse  areas  and  methods  (e.g., 
performance  measurement,  visual 
displays,  instructional  strategies, 
etc.)  are  employed  in  efforts  to 
reduce  training  costs. 

*»  Cost  Reduction  -  The  emphasis  is  on 
reducing  the  costs  of  training  cer¬ 
tain  skills  to  a  specified  minimum 
level,  as  opposed  to  enhancing 
student  performance  (which,  neverthe¬ 
less,  is  expected  as  a  side  benefit) 
with  associated  increases  in  initial 
training  costs. 

in  line  with  the  project  objective,  begin¬ 
ning  in  1978,  two  prototype  low-cost  aircrew 
training  systems  have  been  conceived, 
designed,  and  fabricated.  Costs  were  $335,000 
for  a  low-cost  cockpit  procedures  trainer 
(LCCF'T)  for  the  SH-3H  aircraft,  and  $800,000 
for  a  low-cost  part  task  trainer  (LCPTT)  for 
the  EA-3B  aircraft.  These  costs  are  approxi¬ 
mately  one-fourth  that  of  comparable  conven¬ 
tional  training  systems.*  Table  1  shows,  for 
each  aircraft,  the  costs  and  designations  of 
the  low-cost  and  conventional  trainers. 


*  The  75  percent  savings  attributed  to  the 
low-cost  systems  is  a  conservative  estimate  to 
allow  for  errors  in  estimating  costs  for  the 
conventional  trairers.  The  estimates  were 
based  on  origins1  system  costs  corrected  for 
inflation  and  costs  of  similar,  more  recently 
developed  trainers. 


119 


TABLF  1  .  COSTS  FOR  LOW  COST 

AND  CONVENTIONAL  TRAINERS 

Aircraf t 

Device 

Cost 

SH-3H 

2C44  (Conventional 
2062  (LCCPT) 

CPT)** 

$1,800,000 
$  335,000 

F.A-  3  B 

2F29  (Conventional 
2C63  (LCPTT) 

OFT)*** 

$4,000,000 
$  800,000 

**CPT  = 
***0FT  = 

Cockpit  Procedures 
Operational  Flight 

Trainer 

Trainer 

The  dollar  amounts  specified  for  the  two 
low-cost  systems  do  not  include  costs  for 
Government  furnished  equipment  (G1E). 
Although  some  GFE  was  used,  GFE  equipment,  as 
opposed  to  simulated  parts,  was  not  considered 
necessary  for  training  effectiveness.  (In 
support  of  this,  many  of  the  components  sup¬ 
plied  Dy  GFE  in  Device  2C63  were  simulated  in 
Device  2C62,  with  no  apparent  loss  in  training 
effectiveness.)  The  GFE  that  was  used  was 
included  in  the  low-cost  devices  because  it 
was  available  and  desired  by  the  Fleet  Project 
Team. 


GTE  in  Device  2C62  consisted  only  of  the 
throttle  quadrant,  which  was  used  instead  of 
modifying  a  simulated  throttle  quadrant,  which 
was  originally  provided  with  the  trainer.  All 
else  was  simulated.  GFE  for  Device  2C63 
included  all  panels  and  inactive  switches, 
some  active  switches,  throttle  and  throttle 
quadrant,  yoke  and  rudder  pedals.  Simulated 
equipment  corsisted  of  some  active  switches, 
all  instruments,  all  wiring  and  the  shell. 
Additional  costs  fcr  simulating  the  components 
which  were  provided  for  by  the  GFE  are  esti¬ 
mated  at  approximately  $200,000  for  Device 
2C63  and  $10,000  for  Device  2C62.  These  costs 
were  not  included  in  the  prices  for  the  two 
low-cost  devices  because  the  prices  given  for 
the  conventional  trainers  also  do  not  reflect 
GFE  costs.  These  unaccounted  costs  would  be 
expected  to  be  at  great  or  greater  for  conven¬ 
tional  systems  than  for  the  iow-eost  systems, 
because  an  effort  was  made  to  discourage  use 
of  CFE  part  1  for  the  low-cost  systems  for  pur¬ 
poses  of  the  R&D  project.  The  seventy-five 
percent  savings  claimed  for  the  low-cost 
systems  is  valid,  however,  even  if  these  addi¬ 
tional  GFE  costs  are  added  to  the  low-cost 
systems  costs  and  not  to  those  of  the  conven¬ 
tional  trainers. 

Development  times  for  the  low-cost  systems 
range  between  one  to  two  years  versus  two  to 
four  years  required  for  conventional  pro¬ 
grams.  Although  not  fully  demonstrated  yet, 
the  modular  and  simpler  low-cost  design  is 
expected  to  produce  fewer  maintenance  prob¬ 
lems,  with  reduced  repair  time  and  costs. 

Additional  life-cycle  savings  will  be 
realised  from  less  expensive  facility  require¬ 
ments  for  low-cost  systems.  Also,  instructor 
involvement  with  the  training  process  has  been 


reduced  34  percent  in  a  preliminary  effort, 
and  further  reductions  in  instructor  time 
(along  with  improved  trainee  performance)  are 
expected  with  improvements  in  device  utiliza¬ 
tion  procedures.  This  reduction  saves  person¬ 
nel  costs;  hut,  more  importantly,  it  allows 
greater  utilization  of  the  limited  number  of 
available  instructors  for  other  important 
tasks . 


DESIGN  CHARACTERISTICS  AND  CONCEPTS 
Design  Characteristics 

The  low-cost  systems  consist  of:  (1) 
Device  2C62  (see  Figure  1),  an  LCCPT  designed 
to  provide  training  for  all  normal  and  emer¬ 
gency  cockpit  procedures  as  performed  in  the 
SH-3H  helicopter;  and  (2)  Device  2to3  (see 
Figure  2),  an  LCPTT  constructed  to  train  all 
normal  and  emergency  procedures  and  many 
flight  and  navigation  tasks  required  for  the 
F.A-  3B  aircraft . 


Figure  1.  Device  2C62  (LCCPT) 


Figure  2.  Device  2C6  3  (LCPTT) 
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Detailed  descriptions  of  the  two  low-cost 
systems  may  be  found  in  a  generic  specifica¬ 
tion.  1  The  description  that  follows  is 
intended  to  identify  the  general  nature  of  the 
systems,  their  principal  training  features, 
and  differences  between  them  and  convention¬ 
ally  designed,  higher  cost  counterpart  train¬ 
ing  systems. 

General  Nature.  Each  low-cost  system  has 
three  major  exponents:  (1)  the  cockpit,  with 
controls  and  displays  representing  those  of 
the  real  aircraft;  (2)  the  instructor/student 
station,  providing  a  means  for  problem  initi¬ 
ation,  performance  monitoring,  data  retrieval 
and  computer  programming;  and  (3)  the  computer 
system,  which  activates  and  coordinates  all 
other  systems  of  the  trainer. 

The  simulated  cockpits  contain  functional 
components  which  are  similar  to  the  aircraft 
in  relative  position,  size,  appearance, 
tactile  and  proprioceptive  feel,  and  operating 
characteristics.  Cockpit  displays  significant 
to  training  react  to  student  actions  in  all 
important  respects  just  as  dees  the  actual 
aircraft.  Flight  control  feel  is  simulated 
only  in  Device  2C63  (yoke  and  rudder). 
Appropriate  sound  cues  of  the  aircraft  also 
are  simulated.  Nonfunctional  mockups  were 
used  where  functional  components  are  not 
cost-effective.  These  mociups  duplicate  the 
corresponding  components  ot  the  aircraft  in 
appearance  and  location  only.  As  described 
earlier,  some  Government  furnished  equipment 
was  used. 

Principal  Training  Features.  The  instructor/ 
student  station  contains  keyboard  controls  and 
CRT  displays,  and  is  located  for  convenient 
operation  by  an  ii structor.  The  controls  and 
displays  can  be  rotated  into  position  for 
operation  by  a  student  instructor  or  for  self- 
instruction.  (For  some  training  sessions,  the 
instructors'  normal  interactions  with  trainees 
were  replaced  by  allowing  the  trainee  to  prac¬ 
tice  procedutes  in  the  devices  on  his  ovn  or 
with  the  assistance  of  another  trainee.)  To 
set  up  a  problem,  one  presses  a  key  on  the 
keyboard,  which  automatically  creates  displays 
which  are  appropriate  fer  the  procedure  to  be 
performed.  The  CRT  lists  cockpit  controls 
that  need  to  be  repositioned  manually  before 
the  procedure  begins.  When  these  controls  are 
in  proper  positions  for  the  procedure,  the 
student  attempts  to  perform  the  procedure  in 
the  usual  manner  through  operations  in  the 
simulated  cockpit.  Approx imately  fifty  normal 
and  emergency  NATOPS  (Naval  Air  Training  and 
Operating  Procedures  Standardize ion'  proce¬ 
dures  can  be  practiced  in  this  way  and  over 
100  indiv’duai  aircraft  malfunctions  can  be 
presented  to  the  student  in  a  similar  fashion 
with  each  of  the  two  low-cost  systeti6. 

To  complete  a  Prestart  Checklist  Proce¬ 
dure,  for  example,  the  trainee  perform*  the 
following  steps:  (1)  complete  “Preflight” 

operations;  (2)  chrck  “Upper  Fuel  Caps/Sextant 
Cover/Spoilers  Flaired“;  (3)  assure  that 
“Cabin  Circuit  Breakers”  are  in;  (4)  assure 
that  “LH  or  RH  Fuel  Boost  Circuit  Breaker*” 


are  in;  and  so  on  to  the  end  of  the  proce¬ 
dure.  Procedures  vary  in  length  from  seven 
steps  (e.g.,  a  hydraulic  failure)  to  thirty- 
two  steps  (for  a  start  procedure)  with  many 
actions  required  for  most  steps. 

Errors  made  during  the  execution  of  the 
procedures  are  indicated  by  displaying  on  the 
CRT  or  haru  copy  printout,  the  numbers  and 
names  of  the  procedural  steps  performed 
correctly,  incorrectly  or  omitted  by  the 
trainee,  in  the  order  of  their  performance. 
The  time  for  completing  the  procedure  is 
indicated  on  the  hard  copy  printout  and  the 
CRT. 

To  indicate  a  student's  progress,  a  record 
of  the  performance  of  each  student  instructed 
on  the  system  is  shown  for  each  procedure. 
These  records  include,  e.g.,  the  number  of 
errors  on  the  last  four  trials,  total  trials, 
etc.  Cumulative  totals  are  provided  across  a 
class  to  indicate  group  progress  and  to  allow 
an  individual  to  compare  his  performance  with 
his  classmates.  Examples  of  these  totals 
are:  the  number  of  trainees  who  attempted 

each  procedure,  the  number  of  trainees  who 
achieve  criterion  perf  'rmance  on  each  proce¬ 
dure,  etc. 

The  LCPTT,  in  addition  to  all  NATOPS  cock¬ 
pit  procedures,  allows  training  for  normal  and 
emergency  flight  tasks*  as  indicated  in  Table 

2 . 


TABLE  2.  FLIGHT  TASKS  TRAINED 
WITH  THE  LCPTT 


takeof  fs 

-  takeoff /landing 
emergencies 

climbs 

-  flight  rontrol  boost 
system  failures 

turns 

-  run  away  trim 

cruise 

-  fuel  management  proceoures 

descents 

-  TACAN  and  radio  navigation 

approaches 

-  rad io/communi cat  ions 

procedures  for  crew 
coordination 


-  landings  -  instructor  simulated 

ground  communication 


*  After  corap!e:ion  of  the  training  effective¬ 
ness  evaluation,  the  LCCPT  (Device  2C(2)  wa* 
modified  in  accordance  with  the  instructors’ 
requests  to  include  limited  flight  capabili¬ 
ties.  Altitude,  speed  and  pitch  simulation 
could  be  controlled  with  collective  inputs. 
This  provided  a  task  for  time-sharing  practice 
with  cockpit  procedures,  allowing  trainees  to 
learn  to  perform  the  procedures  and  control 
the  aircraft,  simultaneously. 
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Low-cost  visual  displays  (developed  in 
part  under  a  different  R&D  project  at  this 
Human  Factors  Laboratory)  complement  instru¬ 
ment  displays  with  schematic,  computer  gener¬ 
ated  imagery  of  carrier  and  field  landings  in 
the  practice  of  flight  tasks.  A  low-cost, 
torque-motor  control  loading  system  (adapted 
from  a  similar  system  developed  by  the  Naval 
Air  Test  Center  at  Patuxent  River)  will  soon 
be  implemented  in  efforts  to  increase  the 
fidelity  of  the  "feel"  of  yoke  ard  rudder 
control  movements. 

Differences  with  Conventional  Systems. 
There  are  four  major  differences  between  the 
low-cost  systems  and  their  conventional  coun¬ 
terparts:  (1)  the  low-cost  systems  are  lower 

fidelity  devices  with  respect  to  some  of  their 
components  and  response  characteristics  (see 
next  section  for  examples);  that  is,  the  phys¬ 
ical  similarity  of  the  low-cost  systems  to  the 
actual  aircraft  is  less  than  is  that  of  the 
conventional  systems;  (2)  the  low-cost  systems 
include  simulation  of  engine  and  other  sounds 
associated  with  performance  of  the  training 
tasks,  whereas  conventional  CPT  and  part  task 
trainer  (PTT)  systems  include  no  sound  simula¬ 
tion;  (3)  the  design  of  the  low-cost  systems 
permits  a  limited  sell-  and  peer-instructional 
capability  including  computer  aided  problem 
set-up,  and  automatic  scoring  of  student 
performance^;  and  (4)  commercial  standards 
were  used  for  system  parts  and  documentation. 

Design  Concepts 

The  design  characteristics  described  in 
the  foregoing,  which  are  responsible  for  the 
noted  cost  savings,  are  the  result  ut  consci¬ 
entious  applications  of  rather  pedestrian 
design  concepts.  Generally,  the  design  con¬ 
cepts  indicate  that  training  systems  should 
include:  (a)  only  features  essential  for 

achieving  the  training  objectives;  and  (b) 
instructional  aids  that  facilitate  the  learn¬ 
ing.  Significant  contributions  to  achievement 
of  the  low-cost  goals  are  found  in  day-to-day 
implementations  of  the  low-cost  design  con¬ 
cepts  in  the  face  of  a  variety  of  problems 
associated  with  computer  automation  and  field 
sett i*gs. 

Guidance  for  the  application  of  the  low- 
cost  design  concepts  may  be  found  in  the 
previously  referenced  'eneric  specification. 
More  general  and  extended  guidance  is  being 
developed  4 n  the  forms  of  guidelines  for  per¬ 
forming  the  analyses  that  dictate  the  low-cost 
features. Further  guidance  also  is  being 
initiated,  with  the  support  of  the  Office  of 
Na"al  Research,  in  the  forms  of  systematic 
approach's  to  help  assure  that  particular 
characteristics  of  training  systems,  manage¬ 
ment  procedures ,  procurement  policies  and 
organizational  variables  sre  maximally  con¬ 
ducive  to  the  dcilgn,  acceptance  and  use  of 
the  training  system.  The  importance  of  such 
guidance  it  especially  Important  for  innova¬ 
tive  technology,  as  with  low-cost  systems, 
where  the  goal  is  cost  effective  training, 
rather  than  replication  of  some  operational 


environment.  Without  such  guidance,  training 
programs  are  built  in  accordance  with  far  less 
than  the  best  of  available  technology,  and 
desirable  features  of  systems  are  not  well 
utilized  (see,  e.g.,  Caro,  Shellnut  and 
Spears) . ^ 

Analyses  were  performed  to  include  in  the 
training  system  only  the  minimal  features 
required  to  satisfy  the  training  objectives. 
To  accomplish  this,  discussions  were  held 
among  Human  Factors  personnel,  engineers  and 
subject  matter  experts  in  which  efforts  were 
made  to  determine  whether  certain  cost-saving 
features,  as  listed  in  Table  3,  could  be 
implemented  for  each  training  task,  with  no 
loss  in  training  effectiveness.  These  anal¬ 
yses  resulted  in  simulation  fidelity  levels 
that  are  lower  than  those  of  conventional 
systems . 


TABLE  3.  COST  SAVING  FIDELITY  FEATURES 

-  Elimination  of  redundant  capabilities 

-  Approximate  (vice  exact)  cockpit  dimensions 

-  Chairs  vice  aircraft-type  seats 

-  Photographs  vice  panels 

-  Compressed  instrument  faces 

-  Restricted  needle  movements 

-  Discrete  vice  smooth  needle  movements 

-  S*lk  screen  instrument  faces 

-  Malfunctions  that  give  onset  cues  but  not 
progressive  degradation 

-  Limited  flight  dynamics 

In  reference  to  Table  3,  a  n.alfunction 
needs  to  be  simulated  only  with  one  engine  if 
required  operator  responses  to  the  same  mal¬ 
function  in  the  other  engine  are  the  same. 
Simulation  the  various  engine  malfunctions 
would  be  distributed  among  all  engines,  how¬ 
ever.  This  also  applies  to  hydraulics,  fuel 
tanks,  generators,  etc.  The  approximate  cock¬ 
pit  dimensions  of  tne  low-cost  systems  were 
not  noticeably  different  from  more  exact  (and 
costly)  constructions.  Tasks  could  be  learned 
as  well  using  chairs  instead  of  more  expensive 
seats.  In  many  cases,  photographs  of  a  panel 
were  as  useful  as  more  realistic  panels. 
Graduations  on  instrument  faces  could  be 
compre-sed  imperceptably  and  the  full  range  of 
needle  movement  could  be  reduced  for  some 
tasks  to  help  restrict  needle  movements  to  270 
degrees  (allovtng  the  use  of  a  D’Arsonval 
meter  movement  rather  than  more  expensive 
servo  mechanisms).  Discrete  needle  movements 
could  be  used,  instead  of  smooth  movements, 
where  the  dynamics  of  the  movement  were  not 
important  cues  for  action.  (Trainer  cockpit 
indicators  dc  not  have  to  move  as  far  or  track 
* r*  the  ldeft'Ual  manner  as  the  aircraft  indi¬ 
cators  if  th.*se  characteristics  are  not  essen¬ 
tial  cues,  «s  determined  in  discussions  with 
subject  matter  experts,  for  the  tasks  to  be 
learned.)  Silk  screening  methods  were  less 
expensive  than  using  real  instrument  faces. 
The  simulation  of  a  malfunction  was  t<rs»inateJ 
at  a  point  where  Important  cues  for  action  are 
provided;  all  the  effects  of  inappropriate 
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actions  are  not  provided.  (For  example,  cues 
for  an  engine  fire  are  simulated  without 
including  progressive  degradation  of  the 
system  that  results  from  failure  to  correct 
the  emergency.)  Flight  dynamics  limited  to  60 
degrees  for  bank  and  45  degrees  for  pitch 
saves  money  and  still  were  sufficient  to  pro¬ 
vide  significant  flight  training.  In  these 
cases,  higher  fidelity  would  not  contribute  to 
greater  training  effectiveness;  or  at  least, 
the  contribution  was  not  considered  sufficient 
to  justify  the  higher  costs. 

As  with  any  training  system,  learning  not 
achieved  In  the  low-cost  systems  is  accom¬ 
plished  with  other  .aedia  (e.g.,  classrooms, 
operational-flight  trainers,  aircraft,  etc.) 
where  the  learning  Is  more  cost-effective.  A 
trainee,  for  example,  adjusts  rapidly  to  the 
real  panels  of  the  aircraft  when  trained  with 
pictures  of  panels  that  are  not  directly 
involved  In  the  procedures  *o-be-learned ; 
especially  where,  e.g.,  operational  flight 
trainer  (OFT)  sessions  with  more  realistic 
panels  are  Involved.  It  Is  more  cost  effec¬ 
tive  to  achieve  the  small  amounts  of  learning 
associated  with  realistic  panels  in  the  OFT  or 
aircraft,  because  the  realistic  or  real  panels 
are  required  In  the  systems  for  other  critical 
functions.  The  learning,  therefore,  Is 
accomplished  with  no  additional  development 
costs;  and  because  the  learning  is  rapid, 
increases  in  utilization  costs  (of  the  OFT  or 
aircraft)  are  small. 

Decisions  regarding  the  design  of  "train¬ 
ing  aids”  (e.g.,  automated  performance  moni¬ 
toring,  student  performance  records,  assisted 
problem  set-up,  etc.)  were  based  largely  on 
■‘^eir  ejected  contributions  to  the:  (a) 

c  aeration  of  the  training  system;  (b)  cueing 
of  appropriate  trainee  responses;  and  (c) 
provision  of  useful  performance  feedback  to 
trainees  and  instructors. 

This,  generaliv,  is  the  rationale  for 
designing  the  training  fidelity,  defining  the 
task  components  to-be-trained  with  various 
media  and  providing  instructional  anu  operat¬ 
ing  aids  for  the  two  prototype  systems.  The 
approach  appears  to  be  valid  (see  evaluation 
results  in  the  iollowing  section)  in  the 
current  appl icat ions .  Details  of  the  current 
approach  need  to  be  better  documented  and  its 
cost  effectiveness  needs  continually  to  be 
increased. 

Approximately  50  percent  of  the  noted 
savings  in  development  costs  is  attributable 
to  these  "lideiity"  analyses.  The  remaining 
50  percent  savi.gs  is  due  to  the  use  of  equip¬ 
ment  and  documentation  that  satisfy  but  do  not 
exceed  the  requirements  for  sdnini rtering  the 
training  and  supporting  the  system.  Commer¬ 
cial  (vice  military)  parts  and  standards  wire 
employed  to  obtain  approximately  equal  savings 
for  less  costly  materials  and  less  complex 
documental  ion. 


Device  2C62  Evaluation 

The  acceptability  and  training  effective¬ 
ness  of  the  LCCPT  under  normal  and  modified 
conditions  of  use  have  been  documented. ^ 
Information  was  obtained  from  two  separate 
evaluations  at  two  different  operational  sites 
(HS-1  and  HS-10).  Results  from  the  first 
evaluation  indicated  that  the  LCCPT  does  what 
It  was  designed  to  do.  The  LCCPT  allowed 
training  of  the  same  content,  to  the  same 
level  of  proficiency,  and  with  equal  effi¬ 
ciency  as  the  more  expensive,  conventionally 
designed  counterpart  device.  The  second  eval¬ 
uation  demonstrated  that,  with  proper  utiliza¬ 
tion  procedures,  the  role  of  the  flight 
instructor  when  training  with  the  device  could 
be  reduced. 

The  first  evaluation  consisted  of  a  trans- 
f er-of-training  experiment.  Performances  of 
trainees  who  were  instructed  on  the  low-cost 
device  were  compared  with  performances  of 
trainees  taught  on  the  conventional  device. 
The  comparisons  were  made  both  In  the  trainers 
and  In  thp  aircraft. 

A  savings  of  $1.5  million  was  realized 
with  the  LCCPT  on  development  costs  alone 
($335,000  cost  for  Device  2C62  versus  $1.8 
million  cost  for  Device  2C44),  and  trainee 
performance  in  the  trainers  and  aircrcft  was 
equivalent  for  the  two  systems.  Table  4  shows 
the  time  required  by  trainees  to  achieve 
satisfactory  performance  in  the  aircraft  and 
devices  for  the  low-cost  and  conventionally 
trained  groups.  The  hours-to-prof iciency  in 
the  trainers  and  aircraft  are  In  favor  of 
(i.e.,  lower  for)  the  low-cost  device,  but 
these  differences  do  not  approach  s.atistlcal 
significance. 

TABuF.  4.  TRAININC  TIMES  IN  HOURS 
FLIGHT 


Aircraft  Device- 

Training  Training 

2C62  (LCCPT)  Group 

Mean  15.58  14.33 

S.D.  0.81  1.11 

N  6  6 

2C44  (Conventional 
CPT)  Group 

Mean  16.68  15.81 

S.D.  2.70  2.38 

N  16  16 

Mean  Dif  1.10  1.48 

t  0.97  1.45 

The  f.CCFT  required  modifications  to 
increase  its  simulation  fidelity  for  a  few  of 
its  components  in  order  to  be  acceptable  to 
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instructors  involved  in  the  first  evaluation. 
As  described  previously  and  indicated  in  Table 
4,  the  lower  fidelity  levels  appear  not  to 
have  degraded  critical  task  performance.  In 
order  to  adapt  to  the  lower  fidelity  of  the 
new  device,  the  instructors  did  modify  their 
normal  instructional  methods.  The  Instructors 
emphasized  to  the  trainees  operational  cues 
that  were  missing  in  the  device  in  order  to 
achieve  the  high  standards  reflected  in  the 
evaluation  results.  This  could  account  for 
the  high  student  performance  in  spite  of  the 
lower  device  fidelity.  This  research  demon¬ 
strates,  at  least  for  the  procedures  moni¬ 
tored,  that  instructors  can  use  lower  fidelity 
devices  to  achieve  training  results  that  are 
equal  to  those  of  higher  fidelity  devices. 
The  LCCPT  was  modified  to  inclu  e  significant 
changes  recommended  by  the  instructors  prior 
to  the  second  evaluation. 

The  instructors  expressed  confidence  in 
the  basic  ability  of  the  LCCPT.  Additional 
validation  of  this  opinion  was  a  contribution 
of  the  second  evaluation.  The  training  condi¬ 
tions  of  the  second  evaluation  were  suffi¬ 
ciently  different  from  those  of  the  first 
evaluation  to  test  the  "robustness"  of  the 
LCCPT,  i.e.,  its  ability  to  continue  to  train 
as  well  under  a  variety  of  operational  condi- 
t ions. 

Experimental  data  on  a  conventional  train¬ 
ing  system  were  not  obtainable  for  comparison 
with  the  LCCPT  in  the  second  evaluation. 
Therefore,  a  detailed  comparison  of  perform¬ 
ance  of  low-cost  versus  conventional  devices, 
as  was  done  in  the  prior  evaluation,  was  not 
repeated.  However,  in  the  second  evaluation, 
the  LCCPT  satisfied  operational  standards  for 
trainee  performance  as  a  replacement  for  an 
Operational  Flight  Trainer  (uevice  2F64B)  in 
svllabus  sections  that  called  for  cockpit 
procedures  training.  All  four  trainees 
received  satisfactory  ratings  in  the  LCCPT. 
Only  one  of  the  four  trainees  failed  a  proce¬ 
dure  in  the  aircraft,  a  normal  occurrence 
according  to  the  Instructors.  This  finding 
extends  the  finding  from  the  first  evaluation 
-  that  the  LCCPT  provides  training  for  cockpit 
procedures  that  Is  the  equal  of  a  convention¬ 
ally  designed  system  -  to  another  situation 
and  another  system.  The  similarity  of  results 
across  the  two  situations  helps  to  establish 
that  the  conclusion  derived  from  the  first 
evaluation  concerning  the  high  training  effec¬ 
tiveness  of  the  LCCPT  does  have  general  valid- 
i  tv. 

The  second  evaluation  was  conducted  to 
determine  whether  the  training  effectiveness 
of  the  LCCPT  as  observed  in  the  first  evalua¬ 
tion  eould  be  extended  to  a  situation  wherein 
peer-  and  se  1  •  -i  n*:t  rurt  ion  are  used  to 
streamline  the  instructors'  interact  ions  with 
trainees.  This  evaluation  shoved  that  some  o! 
the  relatively  costly  and  much  demanded 
instructor  time  could  be  redirected  to  ether 
activities,  with  no  apparent  training  detri¬ 
ment.  A  34  percent  reduction  (10  hours  for 
traditional  approach  versus  b.?>  hours  for 


low-cost  approach)  In  the  time  instructors 
normally  spend  with  trainees,  was  obtained. 
This  reduction,  however,  was  accompanied  by  a 
166  percent  increase  (10  hours  versus  26.6 
hours)  over  previous  syllabus  schedules  in  the 
amount  of  time  the  device  was  used  by  trainees 
(student  voluntary  access  to  the  device  was 
unrestricted).  The  extent  to  which  this 
tradeoff  between  decreases  in  instructor  time 
and  increases  in  device  usage  time  is  neces¬ 
sary  with  the  current  or  any  other  approach  is 
not  known.  Further,  the  extent  to  which  this 
tradeoff  may  have  undesirable  effects  (e.g., 
where  device  time  is  more  scarce  than  instruc¬ 
tor  time)  also  is  not  known. 

The  LCCPT  provided  instruction  for  six 
additional  trainees  from  two  classes  which 
immediately  preceded  the  class  from  which  the 
present  data  were  obtained.  The  performances 
of  trainees  from  these  prior  classes  were  not 
included  In  the  foregoing  analyses  because  the 
peer-  and  sel f-instruct icn  conditions  were  not 
yet  sufficiently  implemented  with  these 
classes  to  test  their  efficacy.  (The 
instructors  needed  to  become  more  familiar 
with  and  confident  about  the  new  device  and 
syllabus  before  integrating  the  peer-  and 
self-instructional  procedures  into  their 
training  routines.)  Thus,  the  data  from  these 
earlier  classes  do  not  reflect  on  the  major 
experimental  issue  of  evaluation  two.  These 
"pilot"  data,  however,  do  provide  additional 
support  for  the  basic  effectiveness  of  the 
LCCPT.  All  trainees  from  these  two  earlier 
classes  passed  all  tests  in  the  trainer  and  in 
the  aircraft;  in  fact,  their  ratings  were 
quite  similar  to  those  of  the  third  class — the 
class  of  major  concern  for  the  second  evalua¬ 
tion.  Mean  performance  ratings  for  trainees 
in  these  two  prior  classes  were  3.05  and  3.06 
in  the  LCCPT  and  3.05  and  3.02  in  the  aircraft 
for  the  first  and  second  classes,  respec¬ 
tively.  These  ratings  are  comparable  to 
ratings  for  the  third,  "experimental"  class, 
i.e.,  3.08  in  the  LCCPT  and  3.02  in  the  air¬ 
craft.  Thus,  these  "pilot"  data  are  consis¬ 
tent  with  the  conclusions  derived  from  the 
other  data  presented  in  this  report  from  both 
evaluations  of  Device  2C62. 

Device  2C63  Evaluation 

A  $3.2  mi l1 ion  savings  ($4  million  cost 
for  a  modern  version  of  Device  2F29  versus 
$800,000  for  Device  2Ch3)  is  estimated  for 
development  of  the  LCPTT  relative  to  the  costs 
for  developing  a  conventional  system  to  train 
the  same  skills.  Although  data  in  trainee 
performance  still  are  not  yet  ready  for  analy¬ 
sis,  the  LCPTT  has  been  providing  training  in 
the  Fleet  (at  VAQ-33,  Key  West)  since  early  in 
1981  to  the  apparent  satisfaction  of  trainees 
and  instructors.  In  addition  to  evaluation  o! 
overall  effectiveness,  special  efforts  will  be 
made  to  evaluate  the  contributions  of  the 
low-cost  visual  and  control  loader  by  coapar- 
ing  the  training  e!  t ect  i veness  of  the  LCPTT 
with  and  without  a  visual  and  with  the  control 
loading  system  versus  a  spring-loaded  control 
svstera. 
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TRANSLATION  OF  BENEFITS 


FUTURE  PLANS 


The  products  from  these  projects  already 
have  changed  some  long-  and  strongly-held 
beliefs  and  attitudes  regarding  training 
system  design  and  use.  Actions  also  are 
changing.  First,  the  low-cost  training 
systems  developed  >,nder  this  R&D  program  have 
been  adopted  to  provide  "valuable  priority 
training..."  (HS-10  message  of  19  Jan  82)  in 
Fleet  applications,  in  accordance  with  the 
experimental  demonstrations.  Second,  the 
savings  demonstrated  for  the  two  low-cost,  R&D 
systems  are  being  translated  into  similar 
savings  for  several  production  training 
systems;  the  costs  of  these  production  models 
represent  significant  breakthrought s  in  train¬ 
ing  system  design.  Of  even  greater  signifi¬ 
cance,  however,  is  the  role  these  projects  can 
play  in  opening  the  door  for  exploration  of 
the  much  greater  potential  that  the  training 
technology  field  appears  to  offer. 

The  prime  targets  for  current  products  are 
CPTs  and  PTTs  developed  by  the  Navy.  Several 
"follow-on"  trainers,*  which  are  largely  based 
on  or  significantly  influenced  by  the  current 
products,  have  been  tasked  for  development  by 
the  NAVTRAEQUIPCEN,  in  addition  to  the  two 
original  prototype  trainers.  In  addition  to 
air  crew  trainers,  one  of  the  follow-on 
trainers  will  teach  driving  skills  for  an 
assault  amphibious  vehicle.  The  contribution 
these  products  can  make  to  still  other  type 
trainers  has  not  yet  been  determined.  It 
appears,  however,  that  significant  contribu¬ 
tions  from  current  products  can  be  made  to  a 
wide  variety  of  trainers. 

In  addition,  low-cost  CPTs  are  being 
developed  for  commercial  use  by  Appl i-Mat ion, 
Inc.,  and  American  Air  Lines  for  a  variety  of 
different  aircraft,  e.g.,  DC-8  (1  unit  for 
Trans  American  Airlines);  S-76  (1  unit  for 
American  Air  Lines);  737  (3  units;  for  CPAIR, 
Gatwick  Training  Center  and  Southwest 
Airlines);  727  (2  units;  for  Mexicans  Air 
Lines  &  Federal  Express).  These  commercial 
training  systems  are  based  on  the  designs  of 
the  two  prototype  systems  developed  under  the 
current  projects  and  show  similar  cost 
savings.  Similar  to  the  two  prototype 
systems,  the  costs  of  the  commercial  and  the 
NAVTRAEQUIPCEN  production  low-cost  develop¬ 
ments  are  approximately  75  percent  less  than 
conventional  counterparts  with,  in  some  cases, 
a  decrease  In  required  instructor  time.  Thus, 
the  current  project  is  responsible  for  an 
approach  to  training  system  design  that  In  its 
ramifications  apparently  is  markedly  changing 
development  practices,  both  in  and  out  of  the 
Navy. 


*  Production  trainers  ut der  development  at  the 
NAVTRAEQUIPCEN  that  are  based  on  and/or 
heavily  influenced  by  the  products  of  this 
project  include  Devices  17a»,i  2Cf»tA,  2Cb4, 
2C61,  2C67.  and  a  CPT  for  the  EA-6B  aircraft 
(device  designation  not  yet  knowi). 


The  current  training  systems  are  consi¬ 
dered  to  be  products  of  relatively  conserva¬ 
tive  applications  of  low-cost  approaches. 
(For  example,  video  disc,  computer  generated 
imagery  and  computer  assisted  instruction 
technology  might  replace  actual  three- 
dimensional  cockpit  simulations.)  Exploit¬ 
ation  of  this  potential  should  involve  defini¬ 
tion  and  demonstrations  of  the  most  cost- 
effective  combinations  of  fidelity  designs  and 
utilization  procedures.  Preliminary  guide¬ 
lines  need  to  be  completed  for  facilitating 
the  application  of  low-cost  approaches  as  well 
as  the  acceptance  of  the  approaches  and  the 
products  by  the  user  community.  Then,  these 
guidelines  need  to  be  tested  and  improved. 
The  desirabi 1 i iy  of  incorporating  the  guide¬ 
lines  into  an  automated  system  developed  at 
this  laboratory  for  aiding  in  the  process  of 
instructional  system  design^.?.^  needs  to  be 
investigated;  and  if  desirable,  the  guidelines 
need  to  be  incorporated. 

The  current  training  effectiveness  evalua¬ 
tion  for  the  LCPTT  needs  to  be  completed,  and 
life-cycle  data  are  needed  on  low-cost  systems 
to  assess  cost-effectiveness  over  the  life  of 
the  systems. 

These  and  other  efforts  are  needed  to  help 
assure  that  the  advances  made  in  the  current 
program  are  not  lost.  Worse  yet,  the  advances 
may  become  human  factors  and  general  training 
setbacks  through  misunderstandings  and  misuses 
of  the  new  approaches.  These  dangers  are 
quite  real  in  that  the  "follow-on"  production 
systems  currently  being  developed  are  based  in 
large  part  on  the  partially  defined  and 
sketchily  documented  low-ccst  approach. 
Training  system  development,  in  all  forms,  is 
a  highly  complex  and  creative  process.  The 
complexity  and  demands  for  creativity  of  rela¬ 
tively  new  approaches,  such  as  the  low-cost 
developments,  are  highly  amplified,  ard  will 
continue  to  be  until  more  of  the  questions 
concerning  low-cost  approaches  can  be  answered 
and  more  of  the  process  becomes  routine. 

Enough  justification  for  low-cost 
approaches',  has  been  provided  by  the  current, 
and  other,  related  investigations  to  encourage 
signifi'ant  investments  of  R&D  resources 
toward  demonstrating  and  improving  the  tech¬ 
nology  and  to  recommend  careful  implementation 
of  low-cost  approaches  in  operational  training 
programs. 
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ABSTRACT 


7.-10  pilots  who  rehearsed  surface  attack  skills  under  high  threat 
conditions  in  a  flight  simulator  survived  a  significantly  higher  proportion 
of  total  RED  FLAG  missions  than  did  pilots  who  did  not  receive  the  simulator 
training.  These  data  support  the  notion  that  simulator  training  may  have  a 
significant  influence  upon  aircrew  survivability  in  high  density  ground 
threat  environments 

INTRODUCTION  ' 


While  several  studies  (Kellogg,  Prather,  and 
Castore,  1980;  Hughes,  Engel,  and  Lidderdale, 
1981)  have  shown  that  it  is  possible  to  obtain 
significant  improvements  in  both  offensive  and 
defensive  skills  under  conditions  of  moderate  to 
high  threat  density  in  a  flight  simulator,  there 
exist  no  data  to  show  that  this  improved  perfor¬ 
mance  transfers  to  the  actual  aircraft  under 
realistic  combat-like  conditions.  The  present 
study  clearly  shows  that,  for  the  case  nf  the 
A-10,  training  in  the  Advanced  Simulator  for 
pilot  Training  (ASPT)  can  produce  significant 
effects  upon  survivability  in  the  operational 
environment.  Furthermore,  given  the  conrtraints 
of  the  present,  study,  it  might  be  assumed  that 
the  potential  benefit  of  such  trailing  may  be 
substantial  indeed. 

METHOD 

SUBJECTS.  Twenty-five  experienced  A-10  instruc¬ 
tor  *pTTots  from  Oavis-Monthan  AF3,  AZ,  served  as 
subjects.  Subjects  had  an  average  of  over  700 
hours  in  the  A-10  and  an  average  of  approximately 
1 500  overall  hours  in  fighter  aircraft. 

APPARATUS.  The  study  was  conducted  on  the  A-10 
configuration  of  the  Advanced  Simulator  for 
Pilot  Training  (ASPT)  located  at  the  Operations 
T-aining  Division  of  the  Air  Force  Human 
Resources  Laboratory,  Williams  AFB,  AZ.  Techni¬ 
cal  references  for  the  device  are  found  in  Gum, 
Albery,  and  Basinger  (1975)  and  in  Rust  (1975). 
Force  cuing  was  provided  through  use  of  a 

g-suit.  G-seat  and  p'atform  motion  cuing  were 
not  in  effect.  A  monochromatic,  computer 
ge^rated  visual  scene  of  a  tier  cal  environment 
was  presented  via  ASPT’s  seven  cathode-ray  tubes 
placed  around  the  cockpit  giving  the  pilot  ♦  110 
degrees  to  -40  degrees  vertical  ruing  and  150 
degrees  of  horizontal  cuing. 

PROCEDURE.  Prior  to  part ic ipatiog  in  RED  FLAG 
8?-£,  eleven  of  the  twenty-five  pilots  were 

trained  in  the  ASPT.  Following  a  brief 

f ami  1 iari zat ion  period,  during  which  tin*  pi  loti 
gained  practice  in  operating  at  low  level  in  the 
A-1C  conf igurat ion  of  the  ASPT,  each  pilot 
received  approximately  two  hours  of  practice  on 
ooth  close  *  i  r  support  and  battlefield 

interdiction  missions. 


Simulated  close  air  support  and  battlefield 
interdiction  missions  were  practiced  in  a 
simulated  electronic  warfare  environment  (see 
Figure  1).  The  threat  array  approximates  that 
of  a  typical  Soviet  air-defense  system  at  the 
Forward  Edge  of  the  Battle  Area  (FEBA).  Field 
elevation  of  the  environment  was  5500  ft  MSL. 
Temperature  was  modeled  as  30  degrees  Centigrade. 
Unlimited  ceiling  and  visibility  were  in  effect. 


Figure  1.  Siriulated  Hostile  Environment 


For  the  close  air  support  (CAS)  mission, 
target  arrays  in  the  simulator  .onsisted  of 
three  groups  of  seven  tank*  ,ach.  Tanks  were 
modeled  to  resemble  the  size  and  appearance  of 
T-6?s.  Targets  for  the  interdiction  mission 
were  the  two  command  posts  (CPs)  located  on  the 
west  side  of  the  valley.  Each  command  post 
'onsisted  of  a  g**oup  of  fou»  vehicles.  Ail 
element*  in  both  the  target  and  threat  arrays 
were  modeled  as  stationary  vehic les/s ites. 
Threat  arrays  consisted  of  ZSU-23-4  antiaircraft 
artillery  e vis,  V.-8,  SA-6,  and  SA-4  surface- 
to-air  missiles.  All  threats  were  modeled  to 
operate  in  an  isolated  (non-netted‘  mode  and 
were  modeled  as  radar  controlled. 
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Following  initialization  from  a  point  just 
outside  the  northernmost  pass  leading  nto  the 
target  area,  subjects  were  free  to  maneuver 
within  the  environment.  and  to  use  whatever 
tactics  they  determined  to  be  appropriate.  No 
instruction  or  direction  was  provided  as  to  what 
tactics  to  use.  Prior  to  entering  the  environ¬ 
ment,  each  subject  was  given  a  verbal  "inter* 
briefing  and  a  map  showing  the  position  of 
suspected  threat  sites.  On  each  trial,  or 
sortie,  the  simulated  A-10  aircraft  was  loaded 
with  1200  rounds  of  30nrt  gun  ammunition.  A 
capability  to  dispense  c.iaff  in  a  manual  (as 
opposed  to  programmed)  mode  was  provided.  Upon 
the  start  of  each  trial,  the  simulator  was 
reinitialized  with  the  full  weapons  load 
described  above. 

Targets  could  be  "ki’led"  by  hits  of  one  or 
more  rounds  from  the  30  im  gun  (simulation  of 
tracers  as  well  as  gun  sound  were  provided). 
When  killed,  a  t'rget  would  momentarily  dis¬ 
appear  from  the  visual  scene  giving  the  pilot 
immediate  feedback  as  to  a  hit.  Although  the 
target  reappeared  following  the  brief  delay, 
subsequent  hits  on  the  target  during  a  trial 
were  no  longer  scored.  Threat  systems  responded 
interactively  to  the  aircraft  in  terms  of  the 
pilot's  use  of  maneuver,  direct  terrain  masking, 
chaff,  etc.  A  "functional"  simulation  of  a  radar 
warning  receiver  (RWR)  was  also  present  in  the 
simulator  cockpit.  The  RWR  symbology  differen¬ 
tiated  between  SAMs  and  AAA.  but  did  not  provide 
specific  symbology  for  the  different  types  of 
SmM  or  AAA.  Unclassified  simulat.ons  of  the 
auditory  cues  associated  with  threat  status  were 
provided  through  the  pilot's  Headset.  At  the 
time  of  the  study,  there  was  no  capability  in 
ASPT  for  simulating  the  AlQ-119  electronic 
countermeasures  pod  (ECM). 

Performance  capabilities  "*f  the  gun  and 
missiles  were  modeled  according  to  unclassified 
sources.  Independent  programs  simulated  the 
aerodynamic  flyouts  of  each  of  the  respective 
missiles.  The  aircraft  was  scored  as  having 
been  killed  if  the  missile  passed  within  50  ft 
of  the  aircraft.  A  visual  image  of  the  missile 
in  flight  appeared  in  the  pilot's  visual  scene. 
However,  no  visual  launch  cues  or  in-flight 
smoke  trait  were  associated  with  missile 
launches.  Muzzle  flashes,  but  no  tracers, 
provided  visual  cues  associated  with  the 
activity  of  the  gun  threat.  In  both  cases, 
"kills"  by  the  threat  resulted  in  the  immediate 
termination  of  tre  trial.  Feedback  was  given 
the  pilot  in  each  instance  as  to  the  conditions 
of  the  kill.  Terrain  crashes  also  caused  a 
trial  to  terminate. 

Following  simulator  training  in  the  ASPT, 
pilots  trained  in  the  simulator  proceeded  to 
Nellis  AFB  where  they  participated  in  RED  FLAG 
8?-?.  Six  of  these  eleven  pilots  flew  the 
exercise  in  A-10  aircraft  equipped  with  the 
AIR-46  Radar  Warning  Receive,  the  ALQ-119  ECM 
pod.  and  witnout  a  chaff  dispensing  capability. 
The  remaining  five  pilots  flew  A-10  aircraft 
equipped  with  the  ALR-69  Radar  Warning  Receiver, 
the  AI.Q  H9  ECM  pod,  and  with  a  ck  iff  dispensing 
capability.  Pilots  in  the  non-simulator  trained 
control  group  were  evenly  distributed  between 


the  two  different  aircraft  configurations.  Data 
from  the  RED  FLAG  exercise  were  collected  on  a 
noninterf erence  basis.  No  changes  or  altera¬ 
tions  to  the  scheduled  range  activities,  scoring 
method',  etc.,  were  made  for  the  study. 

RZSUL1 S 

PRC-RED  FLAG  SIMULATOR  TRAINING.  Although  both 
interdiction  and  close  air  support  were  practiced 
in  the  simulator,  the  following  data  are  for 

close  air  support  only.  In  terms  of  surviva¬ 
bility,  pilots  survived  approximately  25  percent 
of  the  total  sorties  flown  in  the  one  hour  of 

simulated,  close  air  support  training.  Highly 
correlated  with  the  percentage  of  sorties 
survived  was  the  time  each  pilot  was  able  to 
remain  in  the  environment.  There  were  r.o 
constraints  forcing  the  pilot  to  maximize  time 
in  the  environment.  Pilots  were  free  to  exit 

the  target  area  at  wilt.  On  the  average,  pilots 
were  able  to  remain  in  the  environment  for 
between  two  and  three  minutes.  Since  a  "trial" 
was  arbitrarily  terminated  at  the  end  of  four 

minutes,  fhe  brief  duration  of  the  average 
sortie  indicates  that  most  were  terminated 
either  by  threat  kills  or  by  terrain  crashes. 
Mean  time  between  target  kills  was  approximately 
90-seconds  with  the  probability  of  fitting  a 
target  being  about  0.50. 

Approximately  two  thirds  of  all  gun  engage¬ 
ments  occurred  at  altitudes  between  150  and  450 
feet  AGL  at  an  average  range  of  over  4000  feet 
from  the  target.  Figure  2  shows  aircraft  posi¬ 
tion  at  tht  t'T.:-.  it  was  destroyed  by  a  threat  in 
terms  of  altitude  and  range  tu  the  threat  site. 


Figure  2.  Altitude  and  Range  to  Threat  Site 
(  ♦_  1  s.d.1  at  Time  Aircraft  was  destroyed 

The  figure  clearly  shows  that  the  majority  of 
all  kills  here  scored  by  the  AAA.  Thu  frequency 
of  kills  by  the  AAA  in  the  present  simulator 
study  closely  matches  that  observed  for  the  A-10 
during  the  actual  EWCAS  exercise.  The  recorded 
frequency  of  kills  by  the  SA-8  and  SA-6  in  the 
simulator  were,  on  the  other  hand,  low  compared 
to  the  frequency  of  Class  1  miss  distances 
recorded  for  these  threats  in  the  aotuai 
exercise.  This  is  perhaps  due  to  the  50  ft  kill 


128 


radius  employed  in  the  simulator  and  the  150  ft 
kill  ~adius  used  to  defined  a  Class  1  miss  in 
the  actual  exercise.  Figure  3  provides  addi¬ 
tional  data  on  the  bearing  of  the  aircraft  to 
the  threat  at  the  time  the  kill  occurred  in  the 
simulator.  These  data  clearly  show  ttat  the 
aircraft  was  most  o f ten  struck  from  behind. 


Figure  3.  Searing  to  Threat  (  1  s.d.)  at 

Time  Aircraft  was  Destroyed 

The  improvement  in  performance  that  char¬ 
acterized  one  particular  pilot  over  the  course 
of  simulator  training  is  shown  in  Figure  4.  The 
figure  shows  that  following  approximately  4-5 
trials  in  the  simulator,  time  in  the  environment 
(also  siqnJf icantly  correlated  with  sorties 
survred)  began  to  show  a  systematic  increase. 
Figure  4  also  shows,  for  these  same  trials,  a 

steady  decrease  in  the  percent  of  total  time  the 
aircraft  was  within  the  A.AA  ana  SAM  envelopes. 

Equally  as  important  as  those  sorties  where 
the  aircraft  was  killed  by  an  ai»*  def  er  se  threat 
are  those  softies  which  terminated  with  terrain 
crashes.  The  data  in  Table  1  show  tha  .  aside 
from  the  absolute  number  of  crashes  that 
occurred  in  the  simulator,  there  was  ,»  clear 
increase  in  crashes  as  a  function  of  oilot  work 
load.  To  the  extent  tha*  a  simulator  is  unable 

to  capture  all  the  potent » al  sources  of  work 
load  present  in  tne  opt  r  ational  environment, 
these  data  prov  i<>  insight  into  the  extent  to 
which  the  ground,  .tse'f,  may  constitut  *  a 
significant  threat. 

EFFECTS  OF  SIMULATOR  TRAINING  ON  SURVIVABILITY 
AT  RflTElAti  Table  ?  shows  t^e  percent' of  total 
AeD  sorties  survived  as  a  function  of 

whether  or  not  pilots  received  simulator 
training  as  well  as  the  conf igurat ion  of  the 

A- 10  aircraft  flown  during  the  exercise.  These 
data  are  central  to  the  transfer  of  training 

issue  for  flight  simulators.  Two  clear  findings 
are  seen  in  the  data  of  Table  2.  First,  those 
pilots  who  trained  in  a  simulate^  conf4gjred 
like  that  of  the  aircraft  flown  during  th>; 


CONDITION  NOMINA1.  VrtLUt. 

•  FIRING  GUN  DURING  10  SEC  PRIOR  TO  CRASH  <  I  percent 

•  “STRAIGHT  AND  LEVEL”  (less  thin  30°  bank:  less  than 

3g  s:  NO  THREATS  ACTIVE)  i  percent 

•  MANEUVERING  (greater  than  30c  bank;  greate>  than 

3g  s:  NO  ACTIVE  THREAT)  4  percent 

•  MANEUVERINR/ACTiVE  THREAT/S  5  percent 

•  FiflING/MAKtUVERING/ACTIVE  THREAT/S  8  percent 


CR'.SHES  AS  A  PERCENTAGE  OF  TOTAL  SORTIES 
FLOWN  =  19  percent 

Table  1.  Terrain  Crash  Conditions 


AIRCRAFT  CONFIGURATION 


ALR-4C 

AIR-69 

ALQ  119 

ALQM9 

OVERALL 

CONDITION 

NO  CHAFF 

CHAFF 

MEANS 

NO  SIM  TRAINING 

79°  c 

75"  s 

WITH  SIM  TRAINING 

58°  o 

89° » 

m. 

68’ 

82°  o 

Table  2.  Pe; 

rcent  RED 

FLAG  Sorties 

Survived 

as  a  Function  of  Aircraft  Configur 

o  t  ion  and 

Presence/Absence  of 

Simulator  Training 

subsequent  RED 

FLAG  exercise  (i.e.. 

ALR-69  and 

chaff)  survived  a  significantly  larger 
proportion  of  the  total  RED  FLAG  missions  flown 
than  did  those  pilots  receiving  no  simulator 
training.  This  represents  a  clear  positive 
transfer  of  training  effect.  The  second  finding 
shows  equally  as  clearly  that  those  who  trained 
with  the  ALR-69  type  RWR  and  chaff  in  the  simu¬ 
lator  and  who  subsequently  per  ormed  under 
combat-l’ke  conditions  in  aircraft  not  having 
these  capabilities,  performed  significantly 
poorer  than  their  non -si  mu  1  at  or  trained  counter¬ 
parts.  The  effect  of  the  ALQ-110  pod  upon 
survivability  was  net  addressed  by  the  present 
experimental  design.  While  aircraft  configura¬ 
tion  was  not  an  intended  (Manipulation  of  this 
study,  it  clearly  shows  the  powerful  effect  of 
the  simulator  training  upon  subsequent  opera- 
tion'l  performance.  It  also  points  to  the  need 
for  training  aircrews  to  operate  under  worst 
case,  degraded  conditions. 

PIlOT  RESPONSES  TO  QUESTIONNAIRE  ITEMS.  An 
extensive  questionnaire  was  completed  by  pi  lots 
following  the  RED  FLAG  exercise.  A  summary  of 
their  responses  is  contained  in  tne  following 
observations. 

1.  Pilots  were  uniformly  critical  of 
certain  aspects  of  the  simulator's  visual 
system,  specifically  La)  difficulty  of  acquiring 
tarqets  at  extended  ranges,  and  (b)  lack  of 
sufficient  cues  for  flying  low  level  without 
undue  reliance  upon  cockpit  instruments  for 
altitude  r.ference. 
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Figure  4.  Time  in  Environment  as  a  Function  of  Trials  (Left  Figure) 
Percent  Time  in  Threat  Envelopes  as  a  Function  of  Trials  (Right  Figure) 


2.  P'lots  were  in  general  agreement  that 
the  simulated  hostile  environment  contained  the 
critical  cues  found  in  the  tactical  close  air 
support  environment. 

3.  Pilots  estimated  that  thi'  type  of 
simulator  training  might  possibly  improve  the 
survi vabi 1 ity  of  the  current  generation  of  A- 1 0 
pilots  (i.e.,  those  with  little  operational 
experience  and  no  combat  experience)  by  20 
percent  on  the  average.  There  was  a  tendency 
toward  mo'-e  favorable  estimates  by  those  pilots 
having  more  overall  fighter  time  and  more 
experience  in  actual  combat  or  combat-like 
situations. 


CONCLUSIONS 

1.  These  data  provide  empirical  evidence  that 

training  under  high  density  ground  threat 
conditions  in  a  TTight  simulator  can  improve  the 
suryivabTT ity  pF  aircrews  in  a  combat -like 
environment.  *  ’ 

2.  The  fact  that  positive  transfer  of  traini-g 
was  observed  within  the  constraints  of  the 
present  study  (i.e.,  no  formal  training,  per  se; 
limited  training  time  in  simulator;  no  control 
over  content  or  conduct  of  criterion  RED  F LAG 
exercise,  etc.!  suggests  that  Lhe  real  magnitude 
of  this  transfer  of  training  effect  may  be 
substantia’!  Indeed. 

3.  The  occurrence  of  negative  transfer  for 
those  simul ator-trained  crews  who  flew  under 
no-chaff  conditions  in  RED  FLAG  strongly 
indicates  the  need  to  train  crews  for  operation 
under  severely  degraded  or  worst  case  conditions. 

4.  The  unsystematic  use  of  chaff  and  maneuver 


and  tne  fact  that  the  majority  of  all  threats 
struck  the  aircraft  from  the  rear  suggest  that 
serious  training  deficiencies  exist  in  critical 
areas  of  electronic  combat  training. 

5.  The  high  incidence  of  terrain  crashes  has 
serious  implications  for  those  concerned  with 
flight  safety,  especially  under  combat  condi¬ 
tions.  The  present  data  suggest  that  the  ground 
will  present  a  formidable  threat  under  the  work 
load  conditions  of  high  threat.  Vow  level 
tactics. 
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Abstract 


AMTESS  is  the  Army's  attempt  tc  develop  an  operational  model  and  framework  for  acquiring 
modularized,  generic  simulation  systems  for  maintenance  training.  More  broadly,  tne  AMTESS  program  is 
designed  to  leau  toward  a  proven,  systematized,  institutionalized  approach  to  task  analysis,  training 
requirements  v.alysis,  and  fidelity  analysis  in  support  of  training  device  acquisition.  It  is  also 
designed  to  produce  a  model  hardware  configuration  which  includes  a  common  two-dimensional  display 
subsystem  and  a  unique  three  dir. iusional  hardware  subsystem.  Two  prototype  versions  of  the  hardware 
model  which  vary  along  a  number  cf  significant  dimensions  (e.g.,  passive  vs  interactive  use  of  video) 
are  currently  being  evaluated  for  their  transfer  of  training  effectiveness  at  Aberdeen  Proving  Ground, 
Maryland. s_  Transfer  of  training  i  i  being  assessed  on  operational  equipment  using  specially  modified 
versions  of  current  perfc'  Jifr.ci  t-ats,  versions  designed  to  provide  a  rich,  detailed  data  based.  The 
data  base  will  support  asst*  ments  of  overall  prototype  effectiveness  as  well  as  preliminary 
assessments  of  the  effectiveness  of  specific  prototype  features.  The  results  of  these  efforts  will 
support  initial  implementation  of  AMTESS  and  at  the  same  time  will  contribute  towards  a  longer  range 
objective  of  developing  an  operational  model  of  device  acquisition.  l>In  this  paper,  the  AMTESS 
prototypes  will  be  described,  along  with  plans  a no  procedures  for  their  evaluations. 


Introduction 

The  modern  Army  presently  operates  in  an 
environment  characterized  by  complex,  sophisti¬ 
cated  weapon  systems  with  an  increasing  emphasis 
on  support  elements.  In  other  words,  the  Army 
is  laced  with  particular  challenges  in  the  area 
of  systems  maintenance.  Major  changes,  there¬ 
fore  >  are  needed  in  maintenance  training  pro¬ 
grams.  In  response  to  these  changes  and  needs, 
the  military  community  is  responding  by  insti¬ 
tuting  a  large  scale  infusion  of  training 
systems.  One  of  these  is  the  Army  Maintenance 
Training  and  E valuation  System  (AMTESS).  The 
Program  Manager  for  Training  Devices  (PM-TRADE) 
has  been  developing  this  system  as  a  framework 
and  a  mode1  for  future  procurements  of 
mairtenance  trainers,  designed  to  facilitate 
entry  level  training,  as  well  as  to  sustain  and 
evaluate  skill  levels  in  operational  units. 
AMTESS  is  a  modular  system  which  combines  two- 
dimensional  displays  (i.e.,  CRT,  rear  screen 
projectior)  with  three-dimensional,  dynamic 
equipment  aock-ups.  all  linked  to  a  core 
computer. 

The  Array  Research  Institute  (ARI)  has  been 
supporting  the  PM-TRADE  in  this  evaluation  of 
alternative  AMTESS  prototypes  (i.e., 
breadboards)  produced  by  Grumman  and 
Surtek/Seville,  for  training  in  the  following 
MOSs:  Self  Propelled  Artillery  Mechanic  (63D30) 
Hawk  Firing  Sectic  mechanic  (2HC10)  and  Wheeled 
Vehicle  Mechanic  (63W10).  Mini  programs  of 
instruction  have  been  developed  by  Grumman  for 
the  63D30  and  24C10,  and  by  Seville  for  the 
63W10  and  24C10.  The  purpose  of  the  evaluation 
is  to  determine  the  relative,  overall  effective¬ 
ness  of  each  prototype  training  system  compared 
to  training  currently  provided  as  well  as  to 
obtain  student  and  instructor  reactions.  In 


this  paper  discussion  will  be  limited  to  the 
evaluation  for  thr  63D30  and  63W10  special¬ 
ities. 

AMTESS  was  envisioned  as  a  means  of 
applying  advance  simulation  technology  to  a 
family  of  "hands-on,"  "heads-on,"  low  cost, 
self-paced  maintenance  trainers  for  use  at 
installation  and  unit  levels.  The  AMTESS 
concept  was  a  system  to  include:  (1)  actual 
"hands-on"  maintenance  performance  training  for 
specific  maintenance  tasks;  (2)  integration  with 
existing  training  programs;  and  (3)  a  reduction 
in  cost  of  ownership,  or  life  cycle  cost,  for 
both  acquisition  and  sustained  operations. 

Requirements 

To  meet  the  objectives  of  AMTESS,  four 
separate  tasks  were  conducted. 

1.  A  Task  Commonality  Analysis  to  provide 
the  basis  for  selecting  representative  tasks 
within  the  automotive  maintenance  spec u*.Ii ties, 
for  use  in  training  system  design. 

2.  A  Training  Requirements  Analysis  to 
develop  a  :aini  progr  m  of  instruction  for  use  in 
evaluation  of  AMTESS  and  to  demonstrate  the 
feasibility  o^  integrating  the  concept  into 
existing  Army  maintenance  training  programs. 

3.  A  Fidelity  Requirements  Analysis  to 
determine  the  fidelity  requirements  in  a 
training  system. 

4.  A  design  effort  tr  develop  the  concept 
for  AMTESS  and  to  define  a  preliminary  systems 
engineering  design  (PSED)  for  that  concept. 
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Method 


Equipment 

burt::k./seville  -  wheel,  vehicle  mechanic  (63wio) 

Basel  on  the  above  analyses,  Burtek/Sevilie 
Corporation  developed  a  breadboard  model  for 
training  specific  tasks  in  the  Wheel  Vehicle 
Mechanic  (i.e.,  63W10)  speciality.  This  system: 

o  requires  a  trainee  to  use  and  follow 
(i.e.,  perform)  the  activities  presented 
in  the  technical  manuals  designated  fcr 
the  selected  maintenance  tasks. 

o  permits  a  trainee  to  practice  mainten¬ 
ance  tasks  and  obtain  feedback  on 
performance  without  instructor 

supervision. 

o  accommodates  new  training  materials  by 
software  changes  and  appropriate 
preparation  of  new  35mm  slides. 

o  can  be  adapted  to  a  wide  range  of  MOSs 
through  fabrication  of  appropriate 
dynamic  equipment  mock-ups. 

o  includes  a  high  physical  and  functional 
fidelity  equipment  mock-up. 

GENERAL  SYSTEM  DESCRIPTION  -  Burtek/SeviUe3 

The  Burtek/Seville  system  includes  a 
student  station  with  CRT,  rear  projection  screen 
(35mm  slide  projection),  function  keyboard  and  a 
dynamic  equipment  mock-up  (Cummins  Engine). 
These  elements  are  linked  to  an  instructor's 
station  (CRT,  keyboard,  printer)  through  a  16- 
bit,  32,000  word  microprocessor. 

Trainees  are  introduced  to  partic^ar 
exercises  by  the  CRT,  which  then  ref--  s  them  to 
standard  training  manuals  for  detailed 
procedures.  The  rear  projection  screen  is  used 
to  portray  detailed  photographs  of  the  Cummins 
engine  with  indications  of  locations  where 
maintenance  is  to  be  performed.  The  bulk  of  the 
actual  instruction  therefore  is  conducted 
through  hard  copy,  media  (i.e.,  the  TMs  and  35mm 
slides).  The  CRT  does,  however,  play  a  critical 
feedback  role  since  incorrect  actions  on  the 
Cummins  or  on  the  student's  response  panel  (for 
some  simulated  actions)  are  indicated  on  the 
CRT. 

The  instructor  station  includes  the 
controls  and  indicators  necessary  to  manage  the 
program  of  instruction  which  is  delivered  at  the 
student  station.  A  video  terminal  presents 
information  to  the  instructor,  facilitating 
selection  of  training  problems,  selection  of 
systems  failures,  and  malfunctions,  and  presents 
records  of  trainee  performance. 


Training  Management  Programs 

Programs.  The  following  programs  are 
provided  to  guide  the  student  through 
maintenance  procedures  listed  in  his  technical 
manuals,  provide  exercises  in  trouble  shooting, 
and  to  monitor  his  performance: 

o  Training  Exercise  Programs 

o  Failure  and  Malfunction  Programs 

o  Performance  Monitoring  Program 

Training  Exerola-  Fi-uwrams.  Training 
exercise  programs  a.  ;  provided  for  the 
automative  maintenance  trainlre  activities. 
These  prog  -amc  permit  the  lrioi-uctor  to 
initialize  particular  lessons,  introduce  the 
trainee  to  the  lesson,  mor.it-.'-.-  rr.rcific  steps 
and  tneir  sequence  listen  ir.  the  applicable 
maintenance  TMs  for  the  tasks  being  performed, 
and  provide  feedback  to  the  trainee. 

Failure  and  Malfunction  Programs.  These 
programs  control  failure  and  malfunction  for  the 
system  being  simulated.  They  rfc.uc.in  in  effect 
until  corrected  by  the  trainee  or  removed  by  the 
instructor.  The  malfunctions  affect  the 
performance  oi  the  engine  components  and  provide 
appropriate  cues  and  indicators  for  the  trainee 
to  isolate  and  identify  the  faulty  component. 

Performance  Monitoring.  This  program 
provides  monitoring,  sensing  and  recording  of 
performance  errors  of  procedural  steps  related 
to  sp°cific  tasks  included  in  the  simulator. 
Procedural  errors  are  identified,  recorded,  and 
made  available  to  the  instructor  on  CRT  or  hard 
copy  in  an  English  language  text  that  does  not 
require  analysis  or  interpretation.  All 
performance  activities  are  recorded  and 

recallable  by  the  instructor  at  the  instructor 
station  controls. 

Grummar.-Self  Propelled  Artillery  Mechanic 

(63D30) 

Conducting  an  analysis  similar  to  these 
conducted  by  Seville/Burtek,  Grumman  Corporation 
developed  a  breadboard  model  for  training 

specific  tasks  in  the  Self  Propelled  Artillery 
Mechanic  (i.e.,  63D30)  specialty.  This  system: 

o  requires  a  trainee  to  use  and  follow 
(i.e.,  perform)  the  activities  presented 
in  the  technical  manuals  designated  for 
the  selected  maintenance  tasks. 

However,  more  use  is  made  of  tutorial 
instruction  in  the  Grumman  system  than 
in  the  Sevii’ e/Burtek  system.  The 
Grumman  system  employs  color  video  and 
videodiscs  to  present  explanations  and 
demonstrations,  both  written  and  spoken, 
of  how  to  carry  cut  specific  maintenance 
steps. 

o  accomodates  new  training  materials  by 
software  changes,  preparation  of  new 
videodisc  materials. 
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o  can  be  adapted  to  a  wide  range  of  MOSs 
through  fabrication  of  appropriate 
simulated  equipment  components. 

o  includes  high  physical  and  functional 
equipment  component  work-ups.  However, 
components  are  arranged  in  test  bench 
fashion,  and  do  not  replicate  the 
arrangement  on  operational  equipment. 


General  System  Description — Grumman 


The  Grumman  system  includes  a  student 
station  with  a  color  CRT,  CRT  touch  panel  and 
dynamic  equipment  components,  arranged  in  test- 
bench  fashion.  These  elements  are  linked  to  an 
instructor  station  (CRT,  keyboard,  printer) 
through  a  Motorola  68000  microprocessor. 
Programs  and  instructional  materials  are  stored 
on  floppy  and  videodiscs. 


GROUPS 

EXPERIMENTAL 

CONTROL 

Device 

MOS 

Average 

No.  of  GO'S 

Time 

I  Average 

No.  of  GO'S 

Time 

j  Grimmer. 

63D30 

iSurtek/ 
j Seville 

i  _ i 

53W10 

63E30 

1 

J _ 

’ 

L_J 

Figure  1:  Evaluation  Design  for  AHTESS 


Measures 


Trainees  are  led  through  lesson  materials 
by  the  color  CRT  which  gives  instructions  to 
consult  specific  pages  in  the  appropriate 
technical  manuals,  snort  explanations  on 
maintenance  procedures  and  diagnostic 
questions.  Explanations  are  either  written  or 
spoken  and  are  supplemented  by  diagrams  or  video 
demonstrations  of  how  to  perform  specific 
maintenance  tasks.  The  trainee  makes  his 
responses  either  by  touching  a  menu  line  on  his 
CRT  or  performing  some  action  on  one  of  the 
three-dimensional  mock-up  components. 


Training  Management  Programs 


The  information  provided  for  Seville/Burtek 
is  applicable  to  the  Grumman  System  as  well. 


Subjects  and  Design 


subjects  were  chosen  from  two  locations, 
the  Wheeled  Vehicle  Maintenance  School,  in  the 
Edgewood  Area  of  Aberdeen  Proving  Ground  (APC) 
for  the  63W10  MOS  a  :a  the  Ordnance  School  in  APG 
for  the  63D30  MOS.  Because  of  a  recent  change 
in  the  training  program  it  was  decided  to  add 
additional  subjects  from  the  Organizational 
Maintenance  Supervisor  (63B30)  training  program, 
also  at  Edgewood  Area.  A  total  of  120  subjects 
were  included  in  the  evaluation,  with  60 
subjects  (i.e. ,  20  in  each  MOS,  63W10,  63D30, 
63B30)  assigned  to  the  experimental  groups 
(i.e.,  training  included  the  AMTESS  devices)  ?::d 
60  subjects  assigned  to  the  control  groups 
(i.e.,  conventional  training  without  AMTESS 
devices) . 


The  performance  measures  typically  used  Ln 
school  programs  have  been  expanded  to  allow  for 
more  detailed  data  collection.  In  addition  to 
the  performance  and  time  measures  collected, 
student,  instructor  and  course  developer’ 
questionnaires  on  each  task  will  also  be 
administered  and  reported. 

Performance  data  will  be  restricted  to 
those  tasks  which  can  be  performed  on  the  actual 
equipment  within  timj  and  safety  constraints. 
These  data  will  consist  of  a  series  of  go-no  go 
decisions  in  a  check-list  format  derived  from 
tire  appropriate  technical  manuals  for  each 
task.  Opinion  or  user  reaction  data  will  also 
be  collected  from  students  and  instructors  upon 
completion  of  training  on  the  devices.  In 
addition,  questionnaire  data  will  be  collected 
from  instructors  and  course  developers  regarding 
those  tasks  which  can  not  be  observed  on  the 
actual  equipment  because  of  time  or  safety 
considerations.  Instructors  and  course 

developers  will  be  presented  with  the  entire 
mini-POI’ s  in  order  to  obtain  training 
efficiency  data.  This  evaluation  will  also 
include  their  reactions  to  determine  if: 

o  tasks  on  each  device  are  also  taught  in 
conventional  training 

o  each  of  the  tasks  are  necessary 

o  each  device  instructs  each  task  to 
acceptable  levels. 

Evaluation  Questions 


An  analysis  of  variance  will  be  conducted,* 
using  the  results  of  a  performance  test 
conducted  for  this  study.  The  dependent 

variables  are  number  of  items  (i.e.,  skills) 
completed  successfully  (i.e.,  measured  by  number 
of  GO's)  and  amount  of  time  to  complete  the 
tasks  and  subtasks.  An  analysis  of  variance 
will  be  conducted  using  the  data  collected  in 

the  following  design  (see  figure  1.  below), 
versus  all  trie  control  subjects).** 


With  the  evaluation  design  constructed  ir 
the  above  manner  the  following  evaluation 
questions  are  anticipated  to  be  answered  by 
using  various  subsets  of  the  data. 

1.  Does  a  simulator  facilitiate 
performance  more  than  conventional 
instruction?  (data  used:  the  average  number  of 
GO’s  and  time  for  all  the  experimental  subjects 
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2.  Does  ?  simulator  reduce  performance 
time  in  relation  to  conventional  training  (data 
used:  average  time  for  all  experimental 
subjects  versus  all  the  control  subjects ) .** 

3.  Is  there  greater  transfer  of  training 
using  a  simulator  than  that  resulting  from 
conventional  training?  (data  used:  average 
number  of  GO'S  for  the  experimental  subjects 
minus  the  average  number  of  GO'S  for  the  control 
subjects,  divided  Dy  average  number  of  GO'S  for 
the  control  subjects). 

4.  Is  there  a  relation  between 
instructor's  and  students'  opinion  about  the 
training  devices  and  students'  performance? 
(data  used:  instructor  ^nd  student 
questionnaire  results  will  be  correlated  with 
average  number  of  GO'S  for  the  experimental 
subjects  only).** 

The  aoove  questions  were  designed  to  provide 
insight,  into  the  overall  effectiveness  of  the 
AMTESS  devices.  The  same  questions  and  data 
indicated  will  be  analyzed  for  each  MOS 
separately.  Because  of  commonality  of  the  tasks 
for  both  experimental  and  control  groups  in  each 
MOS  generalization  of  findings  is  more  likely. 
Restrictions  in  generalization,  therefore,  will 
be  a  function  of  the  reliability,  validity  of 
the  performance  measures,  sample  size,  etc. 

F  rocedures 

All  students,  regardless  of  MOS,  receive 
conventional  training.  Those  students  selected 
to  receive  simulator  training  (i.e.,  on  AMTESS 
devices)  however,  will  be  directed  to  their 
respective  training  d.'/ice  prior  to  conventional 
instruction  on  the  tasks  used  in  this 
evaluation.  The  control  group  will  continue 
with  the  conventional  training  and  be  tested  in 
the  same  manner  as  the  experimental  group  (i.e., 
on  the  actual  equipment).  The  experimental 
group  upon  having  conventional  training  will 
receive  instruction  on  the  device  designed  to 
teach  skills  they  have  not  received  before. 
Both  groups  will  then  be  tested  on  the  same 
skills  or  the  actual  equipment.  The  only 
difference,  therefore,  between  the  experimental 
and  control  groups  will  be  the  use  of  the 
training  device  for  the  experimental  group. 


*  Complete  data  has  not  been  collected  at  the 
time  this  paper  was  prepared,  it  is  assumed, 
however,  all  data  will  be  collected  and  analyzed 
for  the  Presentation  in  November.  The  remainder 
of  this  paper  will  present  the  anticipated 
evaluation  which  will  be  presented  at  the 
conf  erence. 

*•  Generalization  of  these  findings  must  be 
limited  because  of  differences  in  tasks  across 
MOSs. 


Instruction  on  the  AMTESS  devices  will  be 
conducted  by  one  of  the  school  instructors.  The 
following  tasks  were  selected  for  this 
evaluation. 

63W10 

o  Troubleshoot  Engine  Malfunction 
o  Oil  Pump  Filter  and  Pump  Removal 
o  Oil  Pump  Filter  and  Pump  Replacement 

63B30 

o  Adjust  Alternator  or  Drive  Belts 
o  Starter  Motor  Removal  and  Replacement 
o  Oil  Pump  Failure  Troubleshooting 
o  Inspect  Electrical  System 

6;;P30 

o  Starting  System  Problem 
o  i'TM  Setup  and  Checkout 
o  Defective  Transmission  Neutral  Position 
Switch 


Conclusion 

AMTESS  is  conceptualized  as  a  program 
designed  to  acquire  Army  training  devices  by 
systematic  application  of  front  end  analyses. ^ 
The  objectives  of  this  program,  as  delineated  by 
Hofer,^  include* 

o  Development  of  maintenance  trainers, 
utilizing  a  mcdular  format,  for  development  at 
both  institutional  and  unit  levels. 

o  Cost-effective  assessment  methodology 
development  for  Army  Maintenance  training 
programs. 

o  Development  of  Preliminary  Engineering 
models  (i.e.,  breadboard)  to  demonstrate  the 
effectiveness  and  validity  of  the  AMTESS 
concept. 

The  objectives  and  effort  described  here  are 
envisioned  to  be  an  initial  thrust  in  advancing 
Army  ma^nten~nce  training  programs.  It  is 
recognized,  ..owever,  that  gaps  in  knowledge 
still  remain  to  be  filled.  Example  of  such 
information  shortcomings  include: 

o  Generalizability  of  findings 
o  Specific  Device  Architecture 
o  Incorporation  of  AMTESS  into  POI 
o  Research  on  Measures  of  Effectiveness 

Generalizability  of  findings  to  other 
instructional  modules  within  the  MOSs  already 
under  evaluation,  to  other  MOSs,  to  other 
applications,  such  as  use  in  organizational 
settings  or  use  for  skill  qualification 
testing.  At  the  very  least,  implementation  of 
AMTESS  for  the  MOSs  under  present  evaluation 
will  require  the  development  of  additional  POI 
software  modules.  Software  development  for 
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AMTESS  has  proven  to  be  extremely  costly  and 
time  consuming,  even  for  relatively  minor 
changes  in  the  prototype  mini  programs. 
Extension  of  program  development  to  other 
portions  of  the  MOS  POIs  should  therefore  be 
approached  with  great  caution.  It  may  not  be 
necessary  or  even  desireable  to  apply  AMTESS  to 
the  entire  POI.  But  in  that  case  decisions  must 
be  made  about  what  kinds  and  amounts  of 
additional  software  development  are  needed.  The 
proposed  continuing  ARI  research  would  directly 
support  this  decision  making. 

Specific  Device  Architecture.  (both 
software  and  hardware).  This  area  would  focus 
on  such  issues  as:  2D  vs  3D  components  within 
AMTESS;  AMTESS  vs  fiat  panel  devices  such  as 
(EC-3);  and  other  features  such  as  generalized 
troubleshooting  instruction.  The  present  AMTESS 
evaluation  will  not  provide  much  information  on 
the  contribution  of  specific  system  architecture 
to  training  effectiveness.  If  the  prototypes 
proved  to  be  effective,  therefore,  specific 
sources  of  effectiveness  are  not  likely  to  be 
understood.  It  may  be  that  a  relatively  low 
cost  feature  of  a  particular  prototype  system  is 
accounting  for  most  of  the  training 
effectiveness  and  that  some  relatively  high  cost 
feature  could  be  estimated.  The  converse  may 
also  be  '■r’ue,  that  is,  the  addition  of  a  low 
cost  feature  could  dramatically  amplify  the 
effectiveness  of  a  particular  prototype.  For 
example,  evidence  is  accumulating  that  generic 
troubleshooting  training  may  dramatically 
increase  training  system  effectiveness.  Such 
training  could  be  incorporated  into  AMTESS  as  a 
sub-routine  and  would  lend  itself  well  to 
presentation  on  satellite  CRTs  operating  off  an 
AMTESS  main  frame.  This  is  just  one  of  a  number 
of  features  which  would  be  studied  in  further 
ARI  research  under  the  AMTESS  umbrella.  Front 
end  analysis  methodology  needs  to  be  examined. 
Each  of  the  Phase  I  contractors  proposed 
different  breadboard  designs  for  the  AMTESS 
program.  A  methodology  is  needed  whereby  the  PM 
TRADE,  TRADOC,  and  others  can  evaluate  design 
specifications  based  on  sound  guidance  for 
making  har jware  decisions.  That  is,  there  is 
general  agreement  in  conducting  iob/task 
analyses  and  then  training  analyses  in  the 
development  cycle  for  system  requisition.  There 
is,  however,  little  data  on  making  the 
conceptual  leap  from  these  analyses  to  device 
characteristics  deris:  cns. 

Incorporation  of  AMTESS  into  PCI.  Design 
and  use  of  AMTESS  needs  to  be  related  to  system 
variables,  such  as  student  characteristics,  task 
characteristics,  stage  of  training,  use  cf  other 
media,  and  time-based  vs  performance-based 
instruction.  A  number  of  very  serious  issues 
have  already  been  alluded  to.  To  date,  for 
example,  little  or  ro  analysis  has  been  done  on 
how  AMTESS  would  be  incorporated  into  ongoing 
POIs.  For  example,  the  Missile  Maintenance 
Speciality  nas  been  converted  from  self-pacec 
instruction  to  lock-step  instruction.  How  will 
or  should  this  change  influence  the  way  in  which 


AMTESS  is  used?  Student  flow  is  yet  another 
issue  to  be  considered  in  defining  appropriate 
uses  of  AMTESS.  It  was  emphasized  repeatedly  at 
the  la3t  Interservice/Industry  Conference  that 
high  technology  solutions  to  training  which  do 
not  cope  realistically  with  student  flow  are  not 
very  useful  to  the  military.  The  anticipated 
continuing  research  program  will  address  the 
utilization  issue  in  a  major  way  and  support 
both  near  and  long  term  implementation  of  AMTESS 
through  recommendations  to  PM  TRADE  and  the 
TRADOC  on  how  to  effectively  incorporate  AMTESS 
into  ongoing  POIs. 

Research  on  Other  Measures  of  Effectiveness 
such  as  transfer  of  training  to  organizational 
settings,  effectiveness  in  training  "hot"  panel 
repair,  and  use  of  analytic  tools  such  as  the 
TRAINVICE  model.  The  current  AMTESS  evaluation 
involves  measurement  of  transfer  effectiveness 
within  institutional  settings.  The  impact  of 
AMTESS  training  on  transfer  of  training  to  job 
sites  will  remain  unknown. 

Tne  present  evaluation  of  the  Burtek/Seville  and 
Grumman  breadboards  is  an  effort  at  meeting  the 
AMTESS  objectives.  Other  projects  are  planned 
of  AMTESS  prototypes  or  in  progress  which  will 
support  this  program.  A  similar  evaluation,  for 
example,  ir  under  way  in  HAWK  missile 
maintenance  training.  At  the  Air  Defense  School 
in  Fort  Bliss,  an  evaluation  team  is  already  on 
site  collecting  data.  A  supporting  program  of 
basic  research  is  under  way  at  ARI,  in  which  the 
effects  of  simulation  fidelity  upon  training 
effectiveness  is  being  explored.  This  program 
is  described  in  another  presentation  at  this 
conference  (Hays,  1982).  While  these  activities 
represent  considerable  effort  and  progress  in 
meeting  the  AMTESS  goals,  these  must  be  extended 
further. 


References 

1.  Dashow  M.  and  Sicuranz-.  Preliminary  System 
Engineering  Design  Report,  April  198O. 

2  Hays,  R.T.  ARl's  Research  Program  to 
Determine  Training  Simulator 
Characteristics.  Proceedings  of  the 
h  Interservice  Industry  Training 
Equ-.pment  Conference,  16-18  Nov.,  1982. 

3.  Hoi\..  ,  R.C.  The  importance  of  front  end 

analysis  in  training  device 
development.  Paper  presented  at  the 
Society  for  Applied  Learning  Technology 
Conference  on  Front  End  Analysis. 
Arlington,  V A,  July  1981 

4.  Tufsno,  D.  and  Evans,  R.  The  Predictionn  of 

Training  Device  Effective  less,  A  Review 
of  Army  Models.  ARI  Tchnical  Report, 
in  press. 

5.  Whisman,  L.R.  and  Long,  C.E.  Army 

Maintenance  Training  and  Evaluation 


135 


Simulation  System  (AMTESS).  Preliminary 
Syster  3  Engineering  Design.  Technical 
Document,  TD  80-06,  March  1980. 

About  the  Authors 

Dr.  Robert  A.  Evans  is  a  Research 
Psychologist  in  the  Basic  Skills  Instructional 
Systems  Technical  Area  of  the  Army  Research 

Institute  for  the  Behavi'  ral  and  Social 
Sciences.  He  received  his  Doctorate  from  The 
Catholic  University  in  1982.  Prior  to  ARI  he 
was  with  Science  Applications,  Inc.  wnere  he 

developed  the  User's  Guidebook  for  l’RAINVICE. 

Dr.  Angelo  Mirabella  is  Chief  of  the 

Simulation  Systems  Design  Team  in  the  Tiaining 
Technical  Area  of  the  Army  Research  Institute 
for  the  Behavioral  and  oociai  Sciences.  He 

holds  a  Master's  Degree  from  Columc-ia  and  a 
Doctorate  from  the  University  of 
Massachusetts.  Before  coming  to  ARI  he  wao  with 
the  American  Institutes  for  Research. 


ART's  Research  Program  to  Determine  Trainirg 
Simulator  Characteristics 


o 

o 

ft : 


Robert  T.  Havs,  Ph.D. 

Research  Psychologist 

HP  Armv  Research  Institute  for  the  Behavioral  “aid  Foci  a.1. 


*  '~^rhe  Armv  Research  Institute  for  the  Behavioral  and  Pocial  Pciences  ( APT )  is  developing  a 
data  base  or  the  relationship  between  training  simulator  charateristics  -  in  particular  fidelity 
-  and  training  effectiveness.  Tn  order  to  guide  and  organize  the  collection  of  empirical  data 
for  the  data  base,  a  two  factor  definition  of  simulator  fidelity  was  chosen.  The  definition  was 
limited  to  physical  and  functional  similarity  to  the  actual  eouipment.  Several  research 
efforts,  using  this  definition,  are  currently  being  conducted  or  have  just  reached  completion. 
This  paper  presents  the  results,  to  date,  of  completed  research  efforts,  the  anticipated  results 
of  ongoing  efforts,  and  plans  for  future  efforts.  The  goal  of  these  efforts  is  to  produce  a 
data  base  which  mav  serve  as  the  foundation  for  the  development  of  systematic  guidance  to 
support  the  specif icati on  of  training  device  characteristics.^ 


Tntroductior 

'the  Army,  the  other  service*  and  the 
training  community  lack  specific  guidance  to 
determine  the  characteristics  which  will  enable 
training  simulators  to  provide  effective 
training  at  affordable  costs.  In  the  oast  it 
has  too  often  been  the  case  that  the  character¬ 
istics  of  a  training  device  or  simulator  were 
determined  hv  a  mix  of  intuition  (what  someone 
thinks  the  device  should  he  likel  and  cost  (buy 
as  much  "realism”  as  we  can  afford).  This 
approach  to  training  simulator  design  is  no 
longer  a  viable  option  given  budget  and  otter 
resource  constraints.  Accordingly  ART  has 
undertaken  a  program  of  research  to  develop  user 
oriented,  empirically  based  guidance  to  aid  in 
determining  the  characteristics  of  training 
simulators  which  will  maximize  training  effec¬ 
tiveness.  The  program,  from  its  conceptual 
development,  stage,  through  two  ongoing  empirical 
efforts  and  finally  tc  its  future  goals,  is 
;ummari7ed  in  this  paper. 

Defining  training  Simulator  Fidelity 

The  first  step  in  developing  ART’s  training 
itmjlator  research  program  was  to  adopt  a 

working  definition  of  simulator  fidelity.  An 
extensive  literature  search'  '  indicated  a  large 
amount  of  inconsistency  in  how  fidelity  has  been 
defined  hv  the  training  R&r  community.  Tn 
particular,  the  term  has  been  used  somewhat 

indiscrimirisnt  ly  to  lahel  different,  categories 
of  independent  and  decent.  variables  which 
characterize  both  training  svstems  and 
trainees.  It  was  concluded  that  a  limited, 
parsimonious  definition  or  fidelity  would  best 
serve  the  compl  latic;i  of  a  data  base  on  how 

device  characterist ios  are  related  to  training 
effectiveness.  Accordingly,  tne  following 
definition  was  chosen: 

Training  Simulator  Fidelity  is  the  degree 
of  similarity  between  the  training 
simulator  and  the  eaulpment  which  is 
simulated.  It  is  a  two  dimensional 

measurement  of  this  similarity  in  terms  of: 


1)  ,rbe  physioa1  characteristics  of  the 
training  simulator. 


M.e., 

response 


?)  The  functional  characteristics 


the  informational  or  stimulus 
options)  of  the  simulated  eouipment.. 


Tt  was  further  determined  that  for  purposes  of 
empirically  testing  and  validating  this 
definition  as  well  as  for  generating  empirical 
data  on  tbe  relationship  o'*  training  simulator 
fidelity  to  training  effectiveness,  a  three 
level  ordinal  sca'e  would  be  used  for  each 
aspect  of  fidelity  (physical  and  functional). 
Tbi s  approach  yields  the  matrix  like  the  one  in 
Figure  1.  This  nine  cell  matrix  serves  as  the 
Physical  Similarity 


High  Scdiui-i  Low 


Figure  1:  Nine  cell  Matrix  for  Training  iJtor 
Fidelity  Experiments 

"basic”  experimental  design  for  use  in  sub- 
seauent  empirical  efforts. 

Descriptions  of  possible  devices  which 
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would  fit  into  each  cell  of  Figure  1  may  clarify 
this  research  approach.  Cell  HH:  The  actual 
equipment  could  serve  as  the  exemplar  of  a  high 
physical-high  functional  device  and  would  there¬ 
fore  fit  into  the  high-high  cell  of  Figure  1. 

Cell  HM:  The  high  functional -medium  physical 
cell  could  contain  a  fully  functional  device  but 
one  with  some  combination  rf  reduced  size,  number 
and  accuracy  of  its  components  relative  to  the 
actual  equipment.  Cell  HL:  The  high  functional- 
low  physical  cell  could  contain  a  computer  graph¬ 
ics  type  device.  This  device  would  consist  of 
interactive  computer  graphics  which  would  func¬ 
tion  in  an  analagous  manner  to  the  actual  ecuip- 
ment  but  which  would  look  like  line  drawings. 

Cell  MH:  The  medium  functional-high  physical 
cell  could  contain  a  partially  disabled  actual 
device,  in  this  case,  the  device  would  look 
exactly  like  the  actual  equipment,  but  would  be 
only  marginally  functional.  Cell  MM:  A  medium 
functional-medium  physical  device  could  be  bo^h 
partially  functional  and  also  be  degraded  in 
terms  of  size,  number  and  accuracy  of  components 
as  in  cell  HM.  Cell  ML:  The  medium  functional- 
low  physical  cell  could  contain  a  device  which 
consists  of  a  two-dimentional  display  (line 
drawings)  of  the  actual  equipment.  These  draw¬ 
ings  would  afford  the  trainee  a  means  for  indi¬ 
cating  control  choices  O’-  test  points,  but  would 
not  provide  complete  system  responses  to  these 
choices .  Cell  LH:  The  device  tn  the  low  func¬ 
tional-  high  physical  cell  could  be  a  totally 
disabled  piece  of  actual  equipment.  Controls 
would  be  frozen,  displays  and  test  points  would 
be  non  functional.  The  device  would  look  just 
like  the  actual  equipment  but  would  not  work  at 
all.  Cell  LM:  The  low  tunctional -medium  physi¬ 
cal  device  could  be  a  totally  disabled  version  of 
the  degraded  device  used  in  cell  MM.  Cell  LL: 
Finally,  the  low  functional -low  physical  cell 
could  contain  a  device  which  onsists  of  i  set 
of  line  drawings.  These  drawings  wouid  be  phy¬ 
sically  the  same  as  those  in  cells  HL  and  ML,  but 
would  be  totally  non  functional.  These  examples 
are  not  the  only  way  the  nine  cell  matrix  could 
be  filled,  but  they  are  one  concetualization 
which  ARI  hopes  will  be  fillcwed  by  others.  The 
idea  is  to  apply  as  many  alternate  approaches 
to  the  question  of  physical  and  functional  device 
characteristics  and  transfer  ot  training  as  poss¬ 
ible,  but  always  trying  to  maintain  the  nine 
cell  matrix  as  an  organizational  framework. 

The  goal  of  the  empirical  efforts  in  this 
research  program  is  to  provide  data  on  the  rela¬ 
tionship  of  training  device  characteristics 
(fidelity)  tr  training  effectiveness  as  that 
relationship  is  modifiec  by  the  many  training 
system  variables  which  interact  with  fidelity. 
Previous  research  efforts  has  not  attempted  to 
systematically  control  these  Interactive  vari¬ 
ables  but  hav  rather  looked  at  whole  devices  in 
the  context  of  already  established  training  pro¬ 
grams.  ARI  believes  that  it  is  only  by  conduct¬ 
ing  controled,  systematic  experiments  will  we  be 
able  to  generate  the  necessary  data  to  provide 
useful  guidance  to  individuals  who  m;st  specify 
the  characteristics  of  training  devices  and 
programs  of  instruction. 
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Table  1  displays  a  list  of  some  of  the  var¬ 
iables  which  are  believed  to  be  those  which 


taflf  1 


Variable?  Which  Interact.  With  Fidelity 


1.  Tvoe 

-  Operations 

-  Maintenance 

-  others 

?.  Task  Difficulty 

P.  Specific  Skills 

-  Motor  i  . 

-  Perceptual 

-  Cognitive 

-  Others 

n 

1‘.  Trainee  Fonhisti cation 

-  Vcvice 

-  Trtermediate  p. 

-  F Xpert 


Stage  of  Training 

-  introduction 

-  Procedural 

Training 

-  Familiarization 

Trai ning 

-  Transition 

Training 
Training  Context 

-  Institutional 

-  Field 

Trcornoratior  of 
Device  into  POT 

User  Acceptance 

-  instructors 

-  students 


Use  or  Instruc¬ 
tional  Features 

interact  with  fidelitv.  API’s  goal  is  to 
accumulate  data  on  all  of  these  interactions 
(including  data  from  previous  experiments  and 
experiments  conducted  hv  other  research 
organizat ions)  into  a  data  base  which  can  then 
serve  as  the  basis  for  user  oriented  guidance  in 
specifying  the  characteristics  of  b  training 
simulator.  The  first  or  API’s  empirical  efforts 
examined  how  physical  and  functional  fidelity 
were  related  to  training  effectiveness  in  a 
perceptual  motor  task. 


Simulator  Fidelity  in  a  Deroeptual  Motor  Task 


uoreyweli  fDC,  under  contract  to  ApT , 
conducted  an  experiment  to  determine  (1) 
adequacy  of  APT’s  definition  of  simulator 
fidelitv,  (’)  the  appronriateress  of  a  nine  cell 
physical-functional  fidelity  matrix,  and  (?)  t^e 
relationship  of  simulator  fidelity  to  training 
effectiveness  in  a  perceptual  motor  task.  Fau^ 
et  al .  discussed  the  criteria  used  to 
determine  the  task  selected  in  this  experiment. 

1.  The  task  must  embody  the  skills 
required  in  an  actual  maintenance  tasv 
enviroment . 


Task  performance  must  lend  itself  to 
straight  forward  measurement;  the  measuremetns 
must  he  valid,  reliable,  and  sensitive. 

The  task  must  be  learnahle  in  a 
reasonable  period  cr  time. 


-  — . .  -  -  - - -  -■» 
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Based  on  these  criteria  the  task  chosen  for 
this  experiment  was  the  truing  of  a  bicycle 
wheel. 

The  wheel  truing  experiment  used  the 
fidelity  definition  and  0  cell  matrix  discused 
above.  In  this  effort,  only  B  of  the  nine  cells 
were  investigated.  Figure  ?  shows  the  nine  cell 
matrix  with  the  relevant  cells  indicated.  By 

Physical  Similarity 
•Ugh  Medium  Lou 


(Degraded 


•Devitrs  in  these  cells  are  included  In  the  porrentu.il / 
motor  experiment 

Figure  2;  Application  of  the  nine  cell  matrix  in  the 
perceptual/motor  experiment 

comparing  the  results  in  cells  HH,  MM,  anu  LL  the 
general  relationship  of  fidelity  (both  physical 
and  functional)  to  training  effectiveness  was 
determined.  By  cor  paring  cells  HH,  HL,  LH,  and 
LL  an  indication  of  the  relative  contributions  of 
the  physical  and  functional  aspects  of  fidelity 
was  determined. 

The  data  collection  portion  of  the  wheel 
truing  experiment  was  completed  in  June  of 
Detailed  analyses  of  the  data  are  not  available 
at  the  time  of  this  writing,  but  tentative  results 
can  be  presented.  Overall,  there  was  an  improve¬ 
ment  in  t-ainee  performance  under  all  training 
condfr'ons.  Ten  t-tests  were  computed  which 
compared  the  starting  and  finishing  points  in 
each  performance  trial.  Performance  showed 
signif icai.t  Improvement  (p  less  than  .005)  in 
all  conditions. 

In  one  anaivsls,  the  combined  effects  of 
physical  and  functional  similarity  on  training 
effectiveness  were  assessed.  A  one  way  analysis 
of  variance  (ANOVA)  was  used  to  compare  the 
performance,  on  actual  equipment,  of  subjects 
trained  on  the  devices  in  cells  HH,  W,  and  LL 
of  Figure  2.  In  this  analysis  subjects  trained 


in  the  lower  fidelity  conditions  performed 
almost  as  well  as  subjects  trained  in  the  high 
fidelity  condition.  There  was  in  fact  no 
statistically  significant  difference  between 
these  groups.  The  indication  is  therefore  that 
training  devices  for  a  simple  perceptual  motor 
task  may  not  necessarily  need  to  be  designed 
with  high  fidelity. 

Another  analysis  attempts  to  separate  the 
effects  of  the  physical  and  the  functional 
aspects  of  fidelity.  This  analysis  compares 
the  performance  of  subjects  trained  on  devices 
in  cells  HH,  HL,  LH,  and  LL  of  Figure  2.  When 
these  data  are  analyzed  using  a  2x2  factorial 
design,  there  is  no  significant  effect  of  func¬ 
tional  similarity,  but  there  is  a  significant 
effect  of  physical  similarity  (F-4.157;  df *1 ,75 ; 
p  less  than  .05).  In  other  words,  no  matter 
how  the  simulator  functions,  in  this  task  sub¬ 
jects  perform  better  if  the  simulator  is  more 
physically  similar  to  the  actual  equipment. 


These  results  must  be  considered  inconclu¬ 
sive  and  further  analyses  are  reouired.  Though 
the  above  analvsps  are  only  on  terminal  perform¬ 
ance,  additional  analyses  will  compare  the 
subject’s  rate  of  learning  *v  examining  perform¬ 
ance  over  time.  Also  some  form  of  blocking  of 
subjects  may  be  attempted  to  reduce  within  jrroup 
variance.  cven  so,  these  *ata  are  an  important 
first  step  in  validating  the  9  fell  approach  to 
fidelity  research  and  show  that  this  is  a  viable 
method  for  generating  hasio  data  or  the  rela¬ 
tionship  of  simulate.,*  ridelMv  to  training 
effectiveness. 

Generating  fate  on  Tnteractiv»  ,rariaM*-s 

’’’he'  above  perceptual-motor  experiment 
yielded  valuable  data  on  the  relationship  of  the 
physical  and  functional  aspects  of  fidelity  to 
trailing  effectiveness,  but  only  for  that 
specific  task.  Other  tasks  and  other  inter¬ 
ne  ive  variables  reoui  re  further  investima- 
tio".  APT,  throu**  its  basic  research  (A.l) 
program,  is  beginning  to  collect  data  or  twese 
interactive  variables. 

f'ne  P.1  effort,  currentlv  undervav  at  f-eorme 
Mason  Oniversitv,  involves  constructing  both  a 
generic  device  to  use  as  ar  electro-.rechanica! / 
hydraulic  reference  system  and  3everal  degrade*4 
Simula* j'»r5  of  that  reference  system.  mhe  idea 
here  is  ,*hat  the  reference  svstem  will  serve  in 
*he  role  of  actual  eouir^ert.  Tt  will  function 
in  a  variety  or  wavs  such  as  turning  on  pumps, 
meneratinr  *cnes,  turrjrP  or  lights  or  fans, 
rone  of  which  reallv  "do"  anythin*.  “owever 
malfunctions  w’ll  he  introduced  into  the  system 
and  trainees  w*ll  have  to  troubleshoot  and 
repair  it. 
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Different  groups  will  be  trained  on  either 
the  actual  equipment  (the  reference  device)  or 
on  one  o*  several  simulators  of  the  reference 
system  with  varied  degrees  of  fidelity.  The 
performance  of  individuals  from  each  group  will 
then  be  measured  on  the  reference  jevict  to  de¬ 
termine  the  degree  of  training  transfer  for  each 
level  of  fidelity  and  mixture  of  interactive 
variables . 

This  experimental  paradigm  will  provide  an 
opportunity  to  generate,  under  controlled  lab¬ 
oratory  conditions,  the  oasic  data  needed  on 
many  of  the  interactive  variables  in  the  fidelity 
training  effectiveness  relationship  (see  Table  1) . 
Data  collection  will  begin  in  the  Spring  of  1983 
after  the  reference  device  and  simulators  have 
been  constructed.  Systematic  inclusion  of  inter¬ 
active  variables  will  begin  after  initial  nase- 
line  data  have  been  collected.  The  entire  effort 
is  expected  to  conclude  in  the  Summer  of  1985. 

The  Futur*:  Building  a  Data  Base 

The  goal  of  ARI ' s  Training  Simulation  re¬ 
search  program  is  to  produce  a  user  oriented 
guidance  package  to  help  determine  the  character¬ 
istics  which  should  be  incorporated  into  training 
simulators  to  insure  both  training  and  cost 
effectiveness.  It  is  the  strategy  of  this  re¬ 
search  program  to  base  such  a  guidance  package 
on  empirical  data . 

Data  Base  Sources 

Three  major  sources  of  data  will  be  used  to 
construct  the  training  device  characteristics 
data  base.  First,  previously  conducted  studies 
will  be  evaluated  to  determine  whether  the  data 
generated  from  these  efforts  are  suitable  for 
inclusion  in  the  data  base.  If  so,  the  data  will 
ee  categorized  and  added  to  the  data  base.  If 
not  the  particular  study  will  be  categorized  so 
that  additional  efforts  c.v.n  fill  in  the  gap  in 
the  data  base.  At  present,  this  evaluation  of 
previous  research  is  under  way  and  it  is  expected 
to  be  completed  during  the  Fall  of  1982.  An  ARI 
Technical  Report  which  accumul¬ 

ates  and  reviews  the  literature  on  training  de¬ 
vice  research  issues  and  which  will  serve  as  the 
organizing  framework  for  tne  data  base  Is  in  pre¬ 
paration.  This  paper  i-;  expected  to  he  completed 
by  January,  1983, 

A  second  source  of  data  for  the  data  base 
will  coru;  from  ARI's  research  efforts  as  des¬ 
cribed  above.  These  cfforts  will  not  attempt  to 
reinvent  the  wheel  by  duplicating  previous 
studies  which  have  been  deemed  adequate.  They 
will  rather  focus  on  gaps  in  existing  data, 
either  because  previous  efforts  have  not  add¬ 
ressed  the  right  questions,  because  data  are  not 
valid  and  reliable,  or  because  the  results  arc 
not  systematic  enough  for  inclusion  in  the  data 
base.  The  wheel  truing  experiment  described 
above  is  the  first  entry  from  this  data  source. 
Other  basic  (6.1)  and  applied  (6.?)  research 


efforts  are  presently  being  conceived  and  state¬ 
ments  of  work  (SCWs)  are  in  preparation. 

It  is  anticipated  that  enough  data  will  be 
accumulated  by  1984  that  a  preliminary  iteration 
of  the  guidance  package  may  be  produced.  This 
guidance  packae.-  '-'ill  probably  start  as  a  work¬ 
book  bet  will  eventually  be  automated  for  access 
v  i  computer  terminals. 

thp  ^ata  Base 

Tp  on-'pr  to  make  the  training  device 
characteristics  data  base  more  than  just  ar: 
acaderic  exercise,  it  is  recessarv  to  irsure 
that  the  individuals  who  need  guidance  in 
specifying  training  device  oharacterist ics  have 
access  to  the  data  Kr.se.  At  this  time  there  are 
‘■entaMve  plans  to  automate  the  data  base  in 
order  to  allow  users  direct  access  t0  the 
data.  A  preliminary  step  to  automat. pp  fhe  ^ata 
base  i?  tc  organize  it  arround  user  oriented 
research  issues.  This  organizational  effort  is 
curr'"','*v  ur^erwa/  and  a  data  base  framework 
shoul  1  hr  produced  i"  *he  Fall  of  1 Q60. 

Once  a  basic  framework  for  the  1;  “a  base  is 
developed  it  ran  be  use^  as  the  t*asis  for  an 

automate*'  delivery  svstem..  Such  a  svstem,  usi  n<t 
computer  terminals,  would  arrord  access  for 
dec'sion  makers  at  .-arious  no^nt-;  in  tfcp  T*?p 
process.  T4'  would  also  allow  rreouent  updating 
as  new  ard  better  d?*a  are  developed.  Thp 
iterative  nature  of  tbe  prorospd  data  base  can 
insure  t^at  ‘V  tra^ninr  community  will  have  ■  be 
mos4,  recent  data  available  on  a  wide  variety  of 
training  device  research  issues.  With  access  to 
such  data,  the  development  of  training  devices 
reed  not  relv  on  intuition  but  rather  or  t*e 
best  available  i  nforrria*  ion  on  training 
erfec*'iveress. 

rorcl usi or 

ART  is  pursuing  a  broad  research  prmpram 
a4med  ?t  NjiHinp  a  data  base  which  will  relate 
training  simulation  design  character  J  sties,  to 
transfer  of  training  effectiveness.  An  initial 
•'“ramework  for  data  collect  on  built  around  a 
parsimorious  definition  or  simulation  '“idelitv 
has  been  constructed  and  is  beirn?  used  to  puide 
a  series  of  on-roinr  ard  rlanred  laboratory 
experiments.  The  lonp  ranpe  objective  is  to 
'■'reate  a  comprebersi  ve  simulation  de.Ms’n  data 
base  and  to  evolve  that  data  base  into  t.  tool 
which  can  in  speejryinp  tpainine  device 
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ABSTRACT 


Automated  instruction  aids  on  a  training  device  can  significantly  enhance  the  effec¬ 
tiveness  of  the  device.  However,  the  requirements  for  these  aids  ( Ir.structi c nal  Feature] 
must  be  consistent  with  the  intended  use  of  the  trainer.  If  they  are  not  consistent,  the 
system  implemented  may  be  either  more  complex  than  required,  or  totally  inadequate  This 
paper  describes  the  development  of  Instructional  Features  where  this  inconsistency  did 
exist.  In  this  case  the  general  procesiing  requirements  for  student  monitoriny,  student 
feedback,  instructor  reports  and  in  tructor  controls  were  established.  However,  the  speci¬ 
fic  in-classroom  use  of  each  was  not.  When  the  specific  requi remen ts  werp  established, 
they  were  significantly  less  than  the  general  p.ocessing  requirements  implied.  The  system 
design  did  meet  both  the  general  and  specific  requirements.  However,  a  simpler  approach 
would  have  satisfied  the  actual ly-usea  Instructional  Features*  This  .learly  shows  the 
need  to  consider  the  specific  classroom  use  of  InstructionaKFeatures  not  just  the  general 


processing  requirements. 


INTRODUCTION 

This  paper  presents  the  development  of  In¬ 
structional  Features  for  a  Simulated  Aircraft 
Maintenance  Trainer  (SAMT).  The  initial  concept, 
the  software  system  design  and  implementation, 
an  analysis  of  actual  application  versus  the 
original  intent  and  the  factors  to  he  considered 
in  the  application  of  Instructional  Fpatures  to 
other  trainer  systems  are  discussed.  The  ferm 
Instructional  Features  for  this  paper  is  con¬ 
strained  to  means  the  amount  of  automated  moni¬ 
toring  of  a  student's  progress  through  a  task, 
feedback  provided  to  the  student,  post  lesson 
reports  for  the  instructor,  and  instructor  con¬ 
trol  of  information  presented  to  the  student. 

The  trainer  system  for  which  the  Instruc¬ 
tional  Features  were  developed  is  composed  ot  ten 
different  standalone  trainers.  The  ten  tre.ners 
correspond  to  the  following  aircraft  subsystems: 
fire  control,  fligni  control  instruments,  navi¬ 
gation,  electrical,  environmental,  hydraulics, 
weapon  control,  engine  start,  engine  diagnostic 
and  engine  operating  procedures.  Each  trainer 
consists  of  a  master  simulation  control  console 
(MSCC)  and  a  simulation  panel  set  (SFS).  Figure 
1  is  the  block  diagram  for  a  typical  trainer. 

The  MSCC  contains  the  Honeywell  Level  6  computer, 
a  mass  storage  device,  a  lineprinter,  a  CRT  and 
keyboard  and  a  35  MM  projection  system.  The  MSCC 
is  identical  for  each  trainer.  The  SPS  is  unique 
for  each  trainer  and  consists  of  one  or  two 
panels.  On  the  SPS  are  *he  simulated  aircraft 
controls  and  indicators  and  simulated  test  sets 
for  the  aircraft  subsystem  involved.  Graphics 
on  the  panels  provide  for  location  of  the  control' 
and  indicator  on  the  aircraft.  In  addition  to 
the  aircraft  and  test  set  controls  and  indicators, 
there  are  action  and  element  switches  on  the 


i 

panels.  The  action  switches  provide  the  capa¬ 
bility  for  the  student  to  indicate  his  knowledge 
the.  a  certain  action,  such  as  hose/cable  connec¬ 
tion,  operation  of  hand  pumps,,  aircraft  safing 
actions,  etc.,  is  required  as  he  proceeds  through 
a  task.  The  element  switches  provide  to  the 
student  the  capability  to  indicate  which  compo¬ 
nent  he  has  isolated  as  being  faulty  or  to  call 
up  a  35  MM  slide  of  a  particular  component. 

INITIAL  INSTRUCTIONAL  FEATURES  CONCEPT 

The  initial  objectives  for  "he  Instructional 
Features  were  to  support:  (1)  student  testing, 

(2)  lecturing,  (3)  fault  isolation  treeing,  and 
14}  self-directed  learning.  To  accomplish  these 
objectives,  the  Instructional  Features  were  to 
provide:  (1)  monitoring  of  a.  student's  progress 
through  a  tree  structured  sequence,  (21  student 
feedback,  (3)  scudent  testing,  (4)  instructor 
vports,  and  (5)  instructor  control  over  these 
features. 

•  The  student  monitoring  was  to  be  at  a 
level  that  the  trainer  could  uniquely  respond  to 
and/or  record  a  student's  selection  cf  each  branch 
and/or  step  in  a  Job  Guide  (JG)  or  Fault  Isolation 
Manual  (FIM) . 

t  Feedback  to  the  student  was  co  be  a  CRT 
message,  35  MM  slide,  an  audible  sound,  or  any 
combination  of  these.  The  feedback  was  provided 
lo  cue  the  student,  warn  the  student,  or  suspend 
execution  of  the  lesson  as  a  result  of  student 
branch  selection  or  action  taken  on  the  trainer. 

The  cues  providing  instructions  arj  checkout  aids 
required  to  lead  the  student  through  the  training 
exercise  (a  set  of  procedures  from  the  JO  -rd/or 
FIM).  The  warnings  being  for  procedural  and 
operational  errors,  system  malfunctions  and  safety 
violations. 
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Figure  1.  F-16  SAMT  Interface  Block  Diagram 


•  Student  testing  was  the  presenting  of 
questions  with  the  follow-on  sequence  being  de¬ 
termined  by  the  student's  response. 

0  The  instructor  reports  were  to  consist 
of  student  responses  to  test  questions,  procedural 
and  operational  errors  made,  safety  errors, 
actions  taken  on  the  trainee  and  branch  selections 
made. 


ACTION  EVALUATION/RESPONSE  COMMANDS  (CONT.) 


0  Controls  provided  to  the  instructor  were 
to  consist  of  malfunction  selection,  entry  and 
modification  of  plans  of  instructions,  amount  of 
feedback  and  modification  of  testing  materials, 
self-directed  learning  materials  (cues  and 
warnings)  ard  fault  isolation  schemes.  Plans  of 
instructions  were  the  mix  and  ordering  of  student 
testing,  lecturing,  and  fault  isolation  treeing. 
The  amount,  of  feedback  to  be  controlled  by  levels 
of  aiding  selected  by  the  instructor. 

SOFTWARE  SYSTEM  DESIGN 

The  software  system  design  and  implementa¬ 
tion  to  meet  the  Instructional  Features  require¬ 
ments  consisted  of  a  Courseware  Authoring  Lan¬ 
guage  Generator  (CALGEN)  and  an  on-line  real  time 
interpreter  (Procedure  Monitor).  The  actual 
monitoring,  recording  and  feedback  is  specified 
in  the  CAL  which  is  a  high  level  source  language. 
The  source  ’S  input  to  the  CALGEN,  checked  for 
errors  and  decoded  into  an  object  code.  The 
interpreter  decodes  the  CAL  object  code  for  each 
student  action  determining  the  correctness,  pro¬ 
viding  the  student  feedback  and  recording  the 
action. 

The  CAL  was  designed  to  provide  a  friendly 
interface  to  the  training  analyst  developing  the 
step-by-step  training  exercise  for  the  student. 
This  software  (courseware)  being  based  or  the 
step-by-step  actions  called  out  in  the  aircraft 
technical  orders  and/or  job  guides. 


CONTROL  COMMANDS 


NVOKE /MONITOR  NAME 


IVOKE 
F.CE  <^MI 


MONITOR  NAME 


TERMINATE  ^MONITOR  NAME 
IF  <^C0NDITI0N^  THEN 
GO  TO  ^LABEL^ 

ESCAPE  <N> 

WRITER'S  CRAMP  COMMANDS 


^  AT  PRIORITY  ^N^ 

X  to/suspend\ 

V  lu\  RESUME  / 
NAME^ 


ABBRtVIATE  ^TRING^  MEANS^STRING^ 

INCLUDE  <^ILE  NAME^ 

'  LOGICAlX 

MAP  SLIDE  /PHYSICAL  NUMBER  =  NUMBER  \ 
X  LIST  / 


The  basic  language  constructs  are: 
STRUCTURAL  COMMANDS 
PROCEDURE  ^NAME^ 

GLOBAL  VARIABLES  ARE  <UST> 

LOCAL  VARIABLES  ARE  ^LIST^ 

MONITOR  ^NAME^  WATCHING  ^CTION  LIS^ 
END  MONITOR  <^NAME^ 

END  PROCEDURE 


ACTION  EVALUATION/RESPONSE  COMMANDS 

e;:pect  ^action  list^ 

ALLOWING  ^ACTION  IIST^ 

UNLESS  ^CTION  LIST^ 


These  constructs  can  be  used  to  provide 
student  monitoring  with  error  detection,  aiding 
and  equipment  simulation.  An  example  of  this  is: 

L:  EXPECT  SAFETY_LATCH 

EXPECT  SAFETY_LATCH  =  ON  ALLOWING  NONE 
UNLESS  NONE 

IF  CORRECT  THEN  GO  TO  l : SAFF_TO_PROfFED 
END  IF 

(ELSE  THE  WRONG  ACTION  WAS  TAKEN) 

DISPLAY  "Place  safety  latch  on"  AT  CRT 
FOR  LOW  AIDING 

GO  TO  L;EXPFC!j>AFETY_LATCH 

END  EXPECT 

L:  SAFE_T0  PROCEED 

SET  READY  TO  PROCEED  LIGHT  TO  ON 
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L:  SAFE_TO_PROCEED  (CONT.) 

PROJECT  23  FOR  HIGH  AIDING 


CAL  can  be  used  to  provide  student  testing. 
An  example  of  this  is: 

L:  D I SPLAY _ COOL  I NG_A I R_QU EST I ON 

DISPLAY  "Should  cooling  air  be  applied 
before  the  main  electrical 
power  supply  is  hooked  up?"  AT 
CRT 

EXPECT  C _ IRN  =  AN_KYBD_STRING  ALLOWING 

ANY  UNLESS  NONE  END  EXPECT 


IF  AN_KYBD_STRING  =  'NO1  THEN 

DISPLAY  "Error  in  choice  of  cooling 
air  application"  AT  DISK 

DISPLAY  "T.O.  JG-11-00  provides  in¬ 
formation  concerning  the 
application  of  cooling  air. 
Would  it  be  proper  to  recon¬ 
sider  vour  last  answer?"  AT 
CRT 


EXPECT  AN_KYBD_STRING  =  'YES'  GO 
TO  L:DISPLAY_MENU  END  IF 

(A  more  specific  aid  is  needed) 

DISPLAY  "On  page  59  of  T.O.  JG-11-00 
there  is  information  concerning  the 
application  of  cooling  air"  AT  CRT 

GO  TO  L:DISPLAY_MENU 

END  IF 

L:FRONT_PANEL_QUESTION 

DISPLAY  'Should  the  front  panel  be 

removed  before  the  application 
of  the  main  electrical  power 
supply?" 


As  can  be  seen  from  the  language  constructs 
and  by  these  examples,  the  courseware  can  be  con¬ 
structed  to  provide  monitoring  of  student  perfor¬ 
mance  by  comparing  actual  actions  against  a  pre¬ 
defined: 

§  Single  action 


•  Set  of  actions  which  must  be  performed 
sequentially 

•  Set  of  actions  which  may  be  perf 'rmed 
in  any  order 

•  Set  of  actions  which  are  extraneous 
and  to  be  ignored 

•  Number  of  actions 

t  Time 

It  provides  presentation  of: 

•  Simulated  device  responses 

•  Pictorial  material 

•  CRT  messages 

CAL  provides  feedback  for: 

t  Correct  actions 

•  Incorrect  actions 

•  Time 

•  Number  of  actions 

Figure  2  describes  the  courseware  genera¬ 
tion  process  and  the  interface  to  the  on-line 
trainer  software.  The  first  step  in  this  process 
is  the  annotation  of  the  i.O.s  or  Job  Guides. 
Annotation  is  the  specifying  of  the  student  moni¬ 
toring,  feedback  responses,  error  processing  and 
recording  requirements  for  each  step  in  the  T.O.s. 
The  CAL  source  is  coded  based  on  the  annotated 
T.O.s.  CALGEN  is  used  to  compile  the  source 
cede.  The  object  coJe  generated  is  produced  as 
a  set  of  files  on  the  disk  used  by  the  operation¬ 
al  software.  The  object  code  is  interpreted  by 
the  on-line,  real  time  trainer  software  providing 
the  desired  student  monitoring,  feedback  and 
report  generation. 

The  on-line  SAMI  software  system  provides 
two  major  functions:  a  simulation  model  function 
and  a  trainer  control  function  The  simulation 
model  simulates  normal  and  malfunctioning  air¬ 
craft  system  operation  by  supplying  appropriate 
responses  to  simulation  panel  set  (SPS)  inputs. 

The  trainer  control  function  has  overall  control 
of  the  training  function  including  initialization, 
training  exercise  preparation,  training  exercise 
presentation,  monitoring  of  the  student  action 
end  comparison  of  the  action  for  compliance  with 
the  courseware. 

In  addition,  input  and  output  subfunctions 
are  provided  which  condition  and  control  the  data 
flow  to  and  from  the  computer.  A  Powerfail  func¬ 
tion  provides  for  resumption  of  a  problem  exercise 
interrupted  by  a  power  failure.  An  operating 
system  provides  a  multi-tasking  environment, 
standard  peripheral  interfaces  and  file  management. 
A  kernel  component  which  provides  interfaces 
between  the  other  software  components  and  the 
operating  systei.i.  The  Kernel  dispatches  execution 
according  to  priority,  state  of  the  training 
system  and  real  time. 
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Figure  2 

COURSEWARE  PREPARATION  FLOW 


ANNOTATED 

COURSEWARE 

T.O.s 

rnn  t 

_ _ J 

CODING 

UUUillU  W 

SHEETS 

DATA  ENTRY 


Figure  3  shows  the  operational  software 
hierarchy.  The  Instructional  Features  require¬ 
ments  are  satisfied  within  the  trainer  control 
function.  The  control  courseware  provides  ser¬ 
vices  generic  to  the  SAMT.  Training  courseware 
provides  T.O.  unique  information.  The  Procedure 
Monitor  is  the  on-line  interpreter  of  the  course¬ 
ware.  The  training  courseware  specifies  what 
student  inputs  (from  the  panel  or  the  keyboard) 
and  in  what  order  these  actions  are  to  occur. 

The  training  courseware  can  also  specify  what 
response  is  to  be  generated  for  a  given  action. 

The  response  may  be  a  display  on  the  SPS,  a 
CRT  message,  35  MM  slide  and/or  problem  freeze. 
Thus  providing  equipment  simulation,  student 
aiding,  error  feedback  and/or  student  testing. 
Training  courseware  also  specifies  what  informa¬ 
tion  is  to  be  recorded  in  the  student  record. 

The  control  courseware  controls  the  training 
courseware  and  provides  the  instructor  control. 
These  controls  are: 

1.  problem  exercise  list 

2.  message  modification 

3.  status  report  call  up 

4.  freeze  enable 

The  problem  exercise  consists  of  up  to  15 
elements  which  define  an  exercise  to  be  performed 
by  the  student.  These  elements  are: 

1.  procedure  --  Job  Guide  or  fault 
isolation  procedure  contained  in  the  training 
courseware. 

2.  malfunction 

3.  aiding  level  --  The  student  aiding 
level  may  be  low,  medium  or  high. 

4.  parameter  --  simulation  variables  such 
as  temperature,  oil  pressure  and  fuel  quantity. 

5.  Sign  IN  --  Instruction  to  the  student 
to  enter  his  name  and  identification.  This 

i  iformation  is  then  entered  in  the  training 
record. 

6.  time  limit  --  Alloted  time  for  the 
student  to  complete  the  exercise.  If  the  student 
exceeds  this  time,  a  message  is  displayed  and 

the  exercise  halted. 

7.  action  limit  --  number  of  action  steps 
to  be  taken  by  the  student  in  performing  the 
exercise.  If  this  number  is  exceeded,  a  message 
is  displayed  and  the  exercise  halted. 

8.  lesson  --  precanned  problem  exercises 
stored  on  the  disk. 

Thes^  elements  are  entered  by  the  instructor 
in  the  order  he  desires.  The  list  is  then  pro¬ 
cessed  sequentially.  This  allows  the  instructor 
to  select  a  combination  of  malfunctions  and  pro¬ 
cedures  to  be  performed  and  to  speci fy  under 
what  conditions  (aiding  level,  parameters,  time 
and  number  of  actions).  The  lesson  element  pro¬ 


vides  for  routinely  performed  training  sessions 
without  the  instructor  having  to  reenter  all 
elements  each  time.  A  typical  problem  exercise 
might  be: 

1 .  Sign  IN 

2.  aiding  level  high 

3.  temperature  parameter  73° 

4.  action  count  75 

5.  malfunction  1 

5.  procedure  XXX 

7.  procedure  YYV 

8.  malfunction  3 

9.  procedure  AAA 

10.  procedure  BBB 

Items  6  and  9  being  the  operational  check 
procedure  which  would  detect  the  malfunction 
and  items  7  and  10  being  the  fault  isolation 
procedures  for  the  specific  malfunction. 

The  message  modification  control  allows  the 
instructor  to  temporarily  modify  messages  con¬ 
tained  in  the  training  courseware. 

The  status  control  allows  the  instructor 
to  request  display  cf  the  training  record.  The 
training  record  consists  of  the  date  and  the 
sequentially  logged  data  consisting  of  P.E.  list 
item,  state  of  system  freeze,  malfunction, 
aiding  level,  artua1  time,  action  counts,  student 
name,  and  identification  and  items  directed  to 
be  recorded  by  the  training  courseware. 

Freeze  control  allows  the  instructor  to 
enable/disable  halting  of  the  exercise  automati¬ 
cally  when  the  student  commits  a  hazardous  error. 

SPECIFIC  REQUIREMENTS  VS.  CONCEPT 

The  software  system  was  designed  based  on 
the  initial  Instructional  Features  requirements. 
When  the  detailed  Instructional  Features  objec¬ 
tives  (specific  use  of  the  trainer)  were  avail¬ 
able,  it  was  obvious  that  they  were  significantly 
less  than  the  original  concept.  The  specific 
objectives  of  the  student  monitoring  requi rements 
consisted  of: 

1.  critical/hazardous  actions  --  Monitor 
the  warnings  and  cautions  in  the  Job  Guides. 

2.  completion  criteria  --  Monitor  that  a 
few  specific  actions  were  accomplished  by  the 
student  before  allowing  him  to  complete  a  proce¬ 
dure.  The  specific  actions  may  be  procedure 
unique. 

3.  malfunction  removal  prerequisi tes  -- 
these  prerequisites  are  not  procedure  dependent 
but  are  airplane  subsystem  dependent.  It  con¬ 
sists  cf  monitoring  for  certain  actions  being 
accomplished  before  the  student  is  allowed  to 
identify  which  component  he  believes  has  failed. 
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Figure  3.  Software  Hierarchy 


4.  time  monitor  --  Monitor  the  length  of 
time  the  student  required  to  complete  a  procedure 

5.  action  monitor  --  Monitor  the  number  of 
action?  the  student  made  in  completing  a  pro¬ 
cedure. 

The  reasons  for  the  significant  difference 
between  the  initial  concept  of  the  Instructional 
Features  and  the  final  concept  fall  into  three 
areas: 

1.  The  general  requirements  for  the  train 
ing  had  been  determined,  but  the  specific  in¬ 
classroom  use  of  the  trainer  had  not  been  esta¬ 
blished.  The  specific  uses  were  not  determined 
until  the  subject  matter  expert  (SME)  was  asked 
to  define  the  intended  use  of  each  Instructional 
Feature.  This  was  done  to  allow  the  specific 
training  courseware  for  each  procedure  to  be 
specified  and  generated. 

2.  The  full  capability  of  the  Instructional 
Features  was  not  required  by  the  SMEs  for  two 
reasons: 

a)  The  instructors  are  present  during 
the  training  exercise  anc  can  provide  the  aiding 
necessary. 

b)  The  Job  Guides  have  a  high  rate 
of  change  for  a  new  weapon  system,  therefore 
making  it  difficult  to  keep  step-by-step  course¬ 
ware  monitoring  current  with  the  Job  Guide. 

3.  The  cueing  or  student  aiding  require¬ 
ment  was  more  state  dependent  than  Job  Guide 
step  dependent  than  was  originally  thought.  That 
is,  a  cue  is  required  at  the  completion  of  a  task 
or  subtask  at  each  step. 

APPLICATION  OF  INSTRUCTIONAL  FEATURES 

In  some  cases,  the  development  of  a  system 
to  meet  a  larger  spectrum  of  Instructional  Fea¬ 
tures  may  be  desirable.  In  other  cases  this  may 
not  be  true.  There  are  several  factors  which 
must  be  considered  in  making  that  decision. 

Some  of  these  factors  are: 

1.  Is  the  system  for  which  the  maintenance 
trainer  is  being  developed  mature?  In  this  case, 
the  T.O.s  are  available  and  have  a  low  rate  of 
change. 

2.  Is  the  training  to  be  performed  for 
theory  of  general  system  operation?  In  this 
case,  representative  T.O.s  or  procedures  can 

be  used  and  the  changes  of  the  actual  equipment 
T.O.s  need  to  be  incorporated  only  when  they 
represent  changes  in  the  theory  of  operation. 

3.  Is  the  training  objective,  familiari¬ 
zation  with  the  T.O.  system?  Again,  representa¬ 
tive  T.O.s  can  be  used. 

4.  What  is  the  level  of  training  to  be 
provided?  Entry  level  or  basic  training  requires 
more  aides  and  cues  to  assist  the  student  ini¬ 
tially  and  then  gradually  remove  the  crutches. 


5.  Is  it  desired  to  have  uniformity  of 
training,  that  is,  less  dependent  on  individual 
instructors?  The  trainer  providing  at  least  the 
minimum  aiding  and  evaluation  ensures  all  students 
receive  minimum  level  of  instruction. 

6.  Is  there  a  high  student  to  instructor 
ratio?  In  this  case,  both  the  student  and  the 
instructor  need  help  (automatic  student  aiding 
and  monitoring  and  student  performance  reports). 

7.  is  the  requirement  conversion  training, 
that  is,  training  a  B-52  maintenance  man  to  main¬ 
tain  B-l's?  In  this  case,  he  is  familiar  with 
the  T.O.  system.  He  will  group  tasks  rather 
than  performing  them  in  series.  The  student 
does  this  because  he  is  able  to  look  ahead.  For 
example,  he  knows  that  he  can  make  all  connections 
to  a  given  piece  of  equipment  at  once  and  save 
time  rather  than  when  they  are  called  out  in  the 
T.O.  In  this  case,  procedural  aiding  Joes  not 
apply. 

8.  Is  it  desired  to  have  consistent 
training  at  more  than  one  level?  In  this  case, 
where  training  is  to  be  provided  for  entry  as 
wall  as  "conversion  training,"  the  Instructional 
Features  need  to  be  and  can  be  diminishable  and 
tailorable  to  meet  all  levels. 

SUMMARY 

The  system  developed  to  meet  the  initial 
concept  of  Instructional  Features  does  meet  those 
requirements  as  well  as  the  specific  final 
requirements.  This  fact  chows  that  the  Instruc¬ 
tional  Features  for  a  trainer  system  can  be  de¬ 
signed  to  meet  several  applications.  However, 
it  is  obvious  that  a  less  sophisticated  system 
would  have  met  the  final  requirements.  As  can 
be  seen  from  this  example.  Instructional  Features 
must  be  determined  in  two  phases:  1)  determine 
the  general  features  required  by  evaluating  the 
intended  use  of  the  trainer,  and  2)  evaluate  the 
specific  classroom  use  of  each  feature. 
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ABSTRACT 

ThF  paper  presents  a  case  for  greater  utilization  of  automated  performance  measurement 
(ARM)  in  flight  simulators.  The  paper  mentions  the  beginnings  of  APM,  and  describes  in  some 
detail  the  current  variations,  such  as  check. ides,  automissions,  parameter  recording,  proce¬ 
dure  monitoring,  and  others.  It  discusses  tne  attitude  toward  APM  of  different  users,  with 
particular  reference  to  each  of  the  identified  variations.  Finally,  it  presents  conclusions 
designed  to  improve  users'  understanding  of  APM  and  increase  appreciation  of  how  it  can 
assist  instructors^, 

INTRODUCTION 


Performance  measurement  is  one  of  the  more 
challenging  areas  of  flight  simulation.  The 
ability  to  place  students  in  demanding  situations 
and  to  safely  and  accurately  evaluate  how  they 
perform  is  among  a  simulator's  greatest  assets. 

It  should  be  exploited. 

In  an  operational  flight  trainer  (OFT)  or  a 
weapon  system  trainer  (WST)  the  instructor 
observes  the  student's  performance  during  a 
training  exercise  from  either  an  on-board  or  a 
remote  instructor  station.  With  an  on-board 
instructor  station  the  instructor  is  usually 
seated  directly  behind  the  pilot  or  copilot 
(most  on-board  instructor  stations  are  used  in 
simulators  for  multi-place  aircraft)  and  can 
directly  view  the  student's  actions  and  the 
resulting  indications  on  the  instruments  or  on  the 
visual  system  display.  While  this  arrangement 
has  many  advantages,  particularly  in  enabling  the 
instructor  to  detect  hesitation  or  confusion  in 
the  cockpit,  it  leads  to  subjective  evaluations, 
which  in  turn  are  conducive  to  variability  and 
inaccuracy.  Also,  because  all  of  the  instruments 
and  switches  in  the  cockpit  are  not  always  clearly 
visible  to  the  instructor,  he  may  not  be  able  to 
obtain  all  of  the  data  needed  for  precise 
evaluation . 

With  a  remote  instructor  station,  the  in¬ 
structor  is  seated  at  a  console  that  can  contain 
several  CRT's  and  a  visual  system  monitor,  supple¬ 
mented  in  some  cases  by  repeater  instruments.  He 
observes  the  student's  performance  primarily  by 
referring  to  a  number  of  CRT  displays  that  depict 
the  simulated  aircraft's  flight  graphically  and 
numerically.  Many  of  the  on-board  instructor 
station  s  limitations  in  obtaining  data  are 
removed,  but  the  instructor  may  not  be  able  to 
assimilate  and  utilize  the  full  amount  of  informa¬ 
tion  that  is  being  presented  or  is  available. 
Furthermore,  the  problems  of  insuring  objectivity 
and  uniformity  will  remain  if  the  evaluation  of 
the  quality  of  the  student's  performance  is  made 
solely  by  human  judgment. 

In  recent  years  automated  performance 
measurement  (APM)  has  been  used  to  varying  degrees 
to  correct  these  problems.  The  digital  computer 
used  in  modern  flight  simulators  has  a  tremendous 
capacity  for  high  speed,  high  volume  data  pro¬ 


cessing  and  is  able  to  make  timely  and  accurate 
comparisons  of  student-achieved  flight  parameters 
with  predetermined  standards.  Imaginative 
designers  and  creative  programmers  have  produced 
computer  programs  and  CRT  displays  that  present 
the  results  co  instructors  and  students  in  a 
variety  of  ways.  There  is  evidence,  however, 
that  the  potential  of  APM  has  not  been  uniformly 
appreciated  among  the  users  of  flight  simulators, 
and  its  capabilities  are  often  unexploited. 

The  purpose  of  thF  paper  is  to  survey  the 
"state-of-the-art"  of  APM,  from  the  viewpoint  of 
a  contractor  who  builds  flight  simulators  in 
response  to  Government  specifications.  Specif¬ 
ically,  the  paper  will  identify  the  various  uses 
of  APM  found  in  current  simulators,  will  discuss 
the  related  programs,  or  instructional  capabili¬ 
ties,  and  will  examine  the  different  types  of 
displays  that  are  associated  with  these  programs. 
In  addition,  the  paper  will  discuss  tne  opinions 
of  users  regarding  these  versions  of  APM,  and 
will  present  conclusions  and  recommendations 
for  consideration  by  persons  who  are  responsih.e 
for  the  contents  of  specifications. 

VARIATIONS 

Automated  performance  measurement  is  defined, 
for  the  purpose  of  this  paper,  as  the  process  of 
evaluating  student  performance  primarily  through 
use  of  the  computer,  and  expressing  the  results 
in  a  numerical  format.  There  are  many  instances 
in  the  conduct  of  training  in  a  flight  simulator 
where  computer-derived  data  is  used  to  facilitate 
a  human-derived  evaluation  being  made,  but  these 
are  not  considered  to  be  within  the  definition  of 
APM.  The  hallmark  of  APM  is  whether  the  computer 
produces  a  score,  or  a  number  of  scores,  that 
purport  to  express  the  performance  of  the  student. 

Based  on  the  above  definition,  the  origin 
of  APM  can  be  no  farther  in  the  past  than  the 
first  use  of  digital  computers  in  flight  simula¬ 
tors  ,  which  occurred  in  the  early  1960's/*)  It 
has  been  reported  that  Device  ? F90 ,  the  TA-4J 
Operational  Flight  Trainer,  was  the  first  trainer 
to  use  "computer  automated  scoring". (?)  The 
first  unit  of  this  trainer,  which  was  built  by 
Goodyear,  was  delivered  to  the  Navy  in  196q.  A 
slightly  mr re  recent  trainer.  Device  2C24,  the 
UH-1H  Helicopter  Instrument  Flight  Trainer, 
employed  an  adaptive  training  mode  in  which  the 
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computer  evaluated  student  performance  on  certain 
tasks  and  advanced  him  to  higher  difficulty  levels 
based  on  his  proficiency. ( 3 )  This  trainer,  built 
by  Link,  was  also  known  as  the  Synthetic  Flight 
Training  System. 

Currently,  the  uses  of  APM  have  proliferated 
to  the  extent  that  a  number  of  variations  can  be 
found  in  every  simulator.  They  range  from 
complex,  highly  automated  programs  to  simple 
graphic  presentations,  some  of  which,  in  fact, 
do  not  fully  fit  the  definition  of  APM.  The 
following  is  a  description  of  each  type  of  APM 
program,  insofar  as  specific,  unique  types  can 
be  identified,  and  a  discussion  of  its  usefulness 
to  an  instructor. 

Ch eckrid e  s 

Checkrides  can  be  considered  to  be  the 
classical  form  of  APM.  A  checkride  is  a  mission 
or  profile  in  which  the  computer  monitors  the 
student  performance,  usually  from  takeoff  to 
final  landing,  without  intervention  by  the 
instructor.  The  common  use  of  checkride  is  the 
evaluation  of  oroficiency  in  instrument  flight. 

Sometimes  the  term  "programmed  mission"  is 
used  to  describe  a  check-ride.  In  the  EA-6B  VIST 
specification,  for  example,  programmed  mission 
is  used,  but  the  contractor  chose  to  use  the 
term  "computer-evaluated  mission"  instead,  con¬ 
sidering  it  to  be  more  descriptive.  The 
specification  for  the  F/A-18  OFT  calls  for 
"programmed  missions  in  either  the  checkride  or 
automission  mode"  (see  below  for  a  discussion 
of  automi ssions ) . 

An  essential  characterise' c  of  checkrides  is 
the  prevention  of  the  instructor  from  making 
on-line  modifications  to  the  mission,  through 
varying  the  environmental  conditions,  introducing 
malfunctions,  or  tightening  or  relaxing  the 
evaluation  criteria. 

Ideally,  checkrides  should  be  designed  so 
that  the  instructor  is  not  required  to  be  involved 
during  the  mission  other  than  to  start  the 
computer  program.  To  this  end,  checkrides 
usually  consist  of  a  series  of  segments  or  legs 
with  the  end  conditions  of  each  leg  defined  so 
that  the  computer  automatically  advances  to  the 
next  leg  at  the  oroper  time.  During  each  leg 
the  computer  monitors  a  number  of  parameters,  or 
variables,  (heading,  altitude  and  airspeed,  for 
example)  and  determines  whether  the  student  is 
within  the  criteria  for  satisfactory  performance. 
While  the  terminology  for  the  components  of  a 
checkride  is  not  at  all  standardized,  the  value 
of  a  parameter  that  a  student  is  attempting  to 
maintain  is  commonly  called  the  reference  value 
(also  tne  base  value),  the  criteria  for  satis¬ 
factory  accomplishment  of  each  parameter  in  a  leg 
are  stated  as  tolerances,  and  instances  of 
performance  that  are  outside  the  prescribed 
tolerance  are  called  error:  or  deviations. 

Two  types  of  CRT  displays  are  usually  pro¬ 
vided  for  the  instructor,  to  enable  him  to 
monitor  the  progress  of  the  mission.  One  is  a 
map  display  depicting  the  intended  route  of  the 
aircraft  and  the  actual  track.  Usually  the  legs 


of  the  checkride  are  Indicated,  and  a  symbol  is 
sometimes  displayed  beside  the  appropriate  leg 
when  the  tolerance  has  been  exceeded. 

Figure  1  illustrates  one  of  the  map  displays 
for  a  checkride  for  the  CH-53D  OFT.  The  exercise 
is  an  instrument  flight  from  MCAS  Cherry  Point 
to  MCAS  New  River,  North  Carolina.  In  the  lower 
right  corner  of  the  display  is  a  table  reporting 
the  amount  that  any  tolerance  is  being  exceeded 
at  that  instant.  — 

The  other  type  of  display  commonly  provided 
is  an  alphanumeric  description  of  the  leg  current¬ 
ly  being  monitored  plus  at  least  the  next  leg. 

The  specification  for  the  A-6E  OFT  calls  for 
three  legs  to  be  displayed.  The  display  provided 
by  the  contractor,  Sperry,  monitors  the  preceding 
leg,  the  current  leg,  and  the  next  leg.  In  the 
EA-6B  WST ,  also  built  by  Sperry,  four  legs  are 
monitored:  the  preceding,  the  current,  and  the 
next  two  (the  specification  did  not  state  how 
many  were  desired) . 

The  display,  which  has  been  called  by 
various  titles  such  as  the  Scenario  Display, 

Leg  Data  Display,  or  Alphanumeric  Display, 
usually  contains  the  number  of  each  leg  listed, 
a  brief  description  of  the  maneuver  being  accom¬ 
plished  by  the  leg,  the  reference  values  for  the 
monitored  parameters,  the  related  tolerances, 
and  the  end  conditions  for  the  leg.  Sometimes 
a  Remarks  column  is  provided  containing,  among 
other  information,  current  deviations.  Figure  2 
illustrates  the  Scenario  Display  for  the  EA-6B 
WST. 

Designing  a  checkride  so  that  the  computer 
will  infallibly  recognize  the  end  conditions  of 
each  leg  is  an  art.  Scenario  designers  must  guard 
against  a  number  of  pitfalls:  premature  advancing 
of  the  computer  program,  failure  to  advance  when 
the  pilot  commences  a  new  leg,  and  the  charging 
of  invalid  errors^) ,  in  the  final  analysis, 
there  probably  will  never  be  a  fool-proof  check¬ 
ride,  so  perfectly  designed  that  the  computer 
will  be  able  to  cope  with  any  gross  error  that  a 
student  can  commit. 

When  the  computer  scoring  becomes  disasso¬ 
ciated  with  the  flight  profile  of  the  simulator, 
whatever  the  reason,  the  instructor  must  either 
abort  the  mission  or  take  action  to  realign  the 
scoring.  Many  simulators  that  have  a  checkride 
feature  are  equipped  with  a  switch,  sometimes 
labeled  Manual  Advance,  to  enable  the  instructor 
to  advance  the  computer  s_coring  to  the  next  leg. 

In  fact,  the  specification  for  the  A-6E  OFT 
requires  a  "control  to  start  the  next  leg  of  the 
■mission  prior  to  expiration  of  indicated  time", 
presumably  referring  to  a  manual'  advance.  Some 
trainers  have  an  additional  switch  to  retract  the 
computer  scoring  to  the  previous  leg.  Of  course, 
there  should  be  a  way  for  the  instructor  to 
"forgive"  any  leg  with  invalid  errors. 

The  results  of  the  mission  can  be  presented 
to  the  instructor  in  many  ways.  Typically  a 
summary  display  is  provided  that  lists  for  each 
parameter,  as  a  minimum,  the  total  number  of 
deviations  recorded,  the  maximum  deviation,  and 
the  cumulative  time  out  of  tolerance.  The  CEM 
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CEM 

SCENARIO  DISPLAY 

LEG 

NUMBER 

DESCRIPTION 

PARAMETERS 

MONITORED 

LEG  END 
CONDITIONS 

o 

DECELERATE  TO  250  KT 
3  MINUTES  BEFORE 
TIMBER  - 

ALT  20000  +  -  299  FT 
IAS  290  +-  50  KTS 

GT  +-  2  NM. 

DIST  IAF  +  12.5  NM. 

*3 

SPEED  NOW  STABLE  AT 
250  KTS . 

ALT  20000  +-  200  FT 
IAS  250  +-  10  KTS 

GT  +-  2  NM. 

RAD  =-  278  GPO  OR 

DME  =-35  NM. 

4 

TURN  LEFT.  PROCEED 
INDOUND  ON  RAD  278 
GPO .  DESCEND  TO 

8500  FT. 

ALT  14350  +-  5850  FT 
IAS  250  +-  10  KTS 

DME  =-32  GPO 

5 

ESTABLISHED  ON 

RADIAL .  CONTINUE 
DESCENT  TO  8500  FT. 

ALT  14350  +-  5850  FT 
IAS  250  +-  10  KTS 

RAD  278  GPO  +  -  3  DEG 

DME  +-  20  GPO 

Figure  2.  OEM  Scenario  Display  for  EA-6B  WST 
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Summary  Display  for  the  EA-6B  WST  (shown  in 
figure  3}  also  includes  the  total  time  recorded 
and  the  percentage  of  the  total  time  recorded  that 
was  out  of  tolerance.  This  percentage  can  serve 
as  a  score. 

Most  persons  trained  in  statistical  analysis 
prefer  other,  more  sophisticated  methods  of 
evaluating  performance.  The  specification  for 
the  F/A-18  OFT,  for  example,  calls  for  measure¬ 
ment  by  "RMS  deviation  technique"  and  mentions 
"Z  scores".  The  RMS  (root  mean  square)  method 
produces  a  value  which  is  the  square  root  of  the 
sum  of  each  deviation  squared,  thus  it  penalizes 
the  student  (quite  properly)  for  occasional  wide 
deviations.  In  this  method  the  deviation  is 
measured  from  the  arithmetic  mean,  assumed  to  be 
equal  to  the  reference  value,  rather  than  from  a 
tolerance.  Z  scores  compare  a  student's  perform¬ 
ance  with  all  others,  using  a  statistical  measure 
called  the  standard  deviation.  With  Z  scores 
the  user  is  required  to  maintain  a  bank  of 
data  from  all  previous  similar  missions. 

When  these  techniques  were  explained  to 
the  F/A-18  OFT  Fleet  Project  Team  and  other  user 
representati ves  at  a  design  review  meeting, 
the  reaction  was  decidedly  negative.  Apparently 
the  users  considered  that  statistical  approaches 
were  not  suited  to  training  needs.  Consequently, 
the  checkride  in  the  F/A-18  OFT  uses  the  conven¬ 
tional,  tolerance-ori ented  method  of  evaluation. 

The  CH-53  D/E  OFT  also  used  tolerances  in 
its  checkrides,  but  it  has  a  Quality  Control 
Display  and  program  that  enable  the  instructor 
to  compare  any  checkride  results  with  the  "norm" 
of  others  in  the  data  bank.  The  instructor  can 
compare  his  student  with  all  others,  or  with 
groups  of  pilots  with  generally  the  same  total 
hours,  instrument  hour.,,  or  hours  in  the  CH-53 
aircraft.  The  comparisons  are  based  on  the 
percentage  of  time  out  of  tolerance  and  the 
maximum  deviation,  for  each  parameter  per  leg. 


Automissions 

As  used  in  some  specifications,  the  term 
"automission"  refers  to  a  programmed  mission  in 
which  the  instructor  is  allowed  to  make  on-line 
modifications.  The  term  "automated  training 
exercise"  has  also  been  used  to  identify  the 
same  capability.  Neither  term  is  adequately 
discriminating. 

In  some  versions  of  automissions  ie  instruc¬ 
tor  is  permitted  only  to  change  the  environmental 
conditions  and  introduce  malfunctions.  In  others 
he  can  also  change  the  scoring  criteria,  i.e. 
reduce  or  increase  tolerances  so  to  make  the 
parameters  more  difficult  or  less  difficult  to 
maintain. 

Automissions  should  use  the  same  general 
displays  as  checkrides,  i.e.  a  graphic  display 
and  a  scenario  display.  Scoring  should  be  based 
on  tolerances  rather  than  comparison  with  other 
students,  since  the  latter  method  loses  validity 
when  the  instructor  modifies  the  mission. 

Monitored  Maneuvers 

A  unique  approach  for  APM  is  being  developed 
for  the  HU-25A  and  HH-65A  Flight  Training  Systems, 
being  built  for  the  U.S.  Coast  Guard  by  Sperry. 
Rather  than  having  an  hour-long,  or  more,  check¬ 
ride  containing  a  fixed  flight  path,  the 
instructor  will  have  available  a  number  of 
separate  programmed  "maneuvers"  which  he  will  be 
able  to  monitor  via  the  computer  whenever  they 
occur  during  a  normal  training  exercise.  These 
maneuvers  will  have  legs,  parameters,  and  end 
conditions  like  a  checkride;  however,  they  will 
each  be  only  a  few  minutes  in  duration.  They  may 
include  an  instrument  takeoff,  a  holding  pattern, 
a  number  of  published  approaches,  a  GCA,  and  some 
malfunctions  at  critical  times  of  flight.  At  any 
time  during  a  training  exercise,  if  the  student 
is  going  to  perform  one  of  the  monitorable 
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PARAMETERS 

NUMBER 

MAX 

MEASURED 

DEVIATIONS 

+DEV 

ALT 

0 

0.0  FT 

GEAR 

0 

0.0 

FLAP 

0 

0.0  DEG 

RAD 

0 

0.0  RADS 

AO  A 

0 

0.0  UNIT 

GT 

0 

0.0  NM 

IAS 

1 

0.0  KTS 

ROC 

0 

0.0  FPM 

MAX 

CUM.  TIME 

TOTAL  TIME 

% 

-DEV 

OUT 

RECORDED 

0.0  FT 

00:00:00 

00:00:23 

0.0 

0.0 

00:00:00 

00:00:00 

0.0 

0.0  DEG 

00:00:00 

00 : 00:00 

0.0 

0.0  RADS 

00:00:00 

00:00:00 

0.0 

0.0.  UNIT 

00:00:00 

00:00:00 

0.0 

0.0  NM 

00:00:00 

00:00:23 

0.0 

7 . 9  KTS 

00:00:04 

00:00:23 

17.4 

0.0  FPM 

00:00:00 

00:00:00 

0.0 

Figure  3.  CEM  (Checkride)  Summary  Display  for  EA-613  WST 


ILABLE  COPY 


156 


■■ >•*:,**  **»«<”  *i>i?if»^*-^>w^^'!rr. ■<**** &&$**  *■  k 


maneuvers*  the  instructor  will  bo  able  to  initiate 
the  ARM  program  and  obtain  a  computer-derived 
evaluation  of  the  student's  performance  of  the 
ma neuver . 

The  user's  objective  in  specifying  this 
approach  was  to  provide  the  instructor  more 
flexibility  then  ho  would  have  with  a  checkride, 
The  user  was  concerned  that  students  will  become 
familiar-  with  the  sequence  of  maneuvers  in  a 
checkride  thus  reducing  its  validity  as  a 
measure  of  overall  proficiency.  With  the 
mon1'  to  red- maneuver  approach  the  instructor  can 
operate  or  withhold  the  monitoring  program  as  he 
desires,  thus  the  student  does  not  know  when 
ho  is  being  monitored.  The  student  may  learn 
what  maneuvers  are  monitorable  and  therefore  know 
when  he  is  vulnerable,  but  he  can  never  be  sure 
if  the  program  is  operating. 

It  is  interesting  that  the  concept  for  the 
monitored-mancuver  approach  was  first  presented 
at  the  mock-up  conference,  early  in  the  develop¬ 
ment  of  the  simulators.  It  does  not  appear  in 
the  published  specification. 

Senii -Automated  Perfo rmance  Measurement 

Some  specifications  suggest  that  the 
instructor  at  any  time  in  a  training  exercise 
should  be  able  to  enter  parameters  and  tolerances 
for  computer  monitoring.  Since  the  instructor 
will  have  to  control  when  the  monitoring  starts 
and  stops,  this  capability  could  be  called  "semi- 
automated  performance  measuring". 

A  program  to  provide  this  capability  was 
developed  in  1976  for  the  A-4H  and  N  OFT's, 
by  the  Simulation  Engineering  Corporation  (now 
Sperry).  A  CRT  page  (see  Figure  4)  enabled  the 
instructor  to  enter  any  desired  reference  value 
and  allowed  deviation,  or  tolerance,  for  up  to 
15  specified  parameters.  When  entered,  these 
values  appeared  under  REF  VAL  and  DEV  in  the 


STANDBY  column  on  the  page.  When  the  instructor- 
started  the  computer  monitoring,  by  entering  a 
value  for  START  TIME  or  by  designating  START  NOW, 
the  parameter  values  would  appear  in  the  RECORDING 
COLUMN.  The  instructor  could  then  enter  new 
values  in  the  STANDBY  column  in  preparation  for 
the  next  leg,  A  tel etypewriter  automatically 
printed  out  every  entry  made  and  the  time  out  of 
to^rance  plus  the  maximum  deviation  for  each 
parameter  monitored.  If  desired  by  the  instruc¬ 
tor,  a  summary  was  also  printed  reporting  the 
total  time  out  of  tolerance  and  the  total  time 
measured  for  each  parameter. 

This  program  provided  the  ultimate  in  flexi¬ 
bility.  However,  the  demands  on  the  instructor 
were  so  great,  if  he  attempted  to  monitor  a  number 
of  legs  sequentially,  that  he  could  handle  only 
one  or,  at  the  most,  two  parameters  per  leg.  By 
preplanning ,al 1  entries  and  having  an  assistant 
operate  the  keyboard,  an  instructor  could  handle 
possibly  three  or  four  parameters  simultaneously. 
His  role  was  to  determine  precisely  when  to  have 
the  operator  make  the  entries  to  start  and  stop 
monitoring  each  parameter,  a  decision  that  was 
critical  to  the  validity  of  the  scoring. 

Parameter  Recording 

A  number  of  specifications  call  for  auto¬ 
mated  performance  measuring  "on  a  continuous 
basis  as  a  function  of  time",  and  use  the  term 
"parameter  recording"  for  this  capability. 

The  interpretation  is  usually  made  by  contractors 
that  a  strip  chart  recording  is  desired,  although 
specifications  are  customarily  not  that  specific. 
When  both  checkrides  and  parameter  recording  are 
required  on  the  same  simulator,  as  is  the  case  for 
the  EA-6B  WST ,  for  example,  the  contractor  is 
able  to  conclude,  at  least,  that  two  different 
capabilities  are  desired. 

The  parameter  recording  program  in  the  EA-6B 
WST  can  monitor  up  to  18  parameters  s imul taneous- 
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Figure  4.  Page  for  Semi -Automated  Performance  Measuring  (A-4H  and  N) 
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1y,  The  instructor  can  enter  reference  values 
and  tolerances  using  a  CRT  page  (see  figure  5) 
somewhat  similar  to  the  page  for  semi-automatic 
performance  measuring  for  the  A-4H  and  N.  Since 
the  EA-6C  WST  uses  non-page  dependent  formats  for 
data  entry,  an  input  code  is  required  for  each 
parameter,  and  initial,  or  default,  values  are 
provided  for  the  reference  values  and  tolerances. 
The  instructor  can  modify  these  values  as  he 
desires  during  the  exercise. 

The  results  are  continuously  printed  out  on 
a  Versa  tec  printer-plotter.  Each  column  of  the 
printout  has  center-line  that  represents  the 
reference  value  and  two  boundary  lines  that 
depict  the  tolerance  (see  Figure  6).  Some 
parameters,  such  as  landing  gear  and  flaps, 
merely  indicate  an  up  or  down  state. 

The  specification  did  not  require  a  summary 
or  analysis  of  the  printout,  so  if  the  program 
were  extensively  used  during  a  mission,  the 
instructor  would  be  faced  with  considerable  raw 
data  requiring  i nterpretation .  Another  limita¬ 
tion  is  the  fact  that  the  printer-pl otter  cannot 
be  used  for  other  purposes,  such  as  printing  CRT 


displays,  when  the  parameter  recording  program  is 
operating.  This  problem  can  be  corrected  by  add¬ 
ing  another  printer-pl otter  to  the  system— a  solu¬ 
tion  that  would  increase  the  cost  of  the  trainer. 

Most  specifications  are  not  as  ambitious  as 
the  one  for  the  EA-6B  WST.  The  parameter  record¬ 
ing  program  fer  the  A-6E  OFT,  for  example, 
monitors  only  up  to  six  parameters  simultaneously, 
although  the  instructor  can  select  them  from  a 
total  of  19  available.'  The  specification  for  the 
AV-8B  Weapons  Tactics  Trainer  calls  for  recording, 
displaying  and  printing  values  and  tolerances  for 
up  to  ten  parameters,  of  22  available.  The  latter 
specification  stipulates  a  minimum  printout  rate 
of  one  copy  per  five  seconds,  which  implies  that 
a  strip  recording  is  not  necessarily  desired. 

In  the  F/A-18  OFT  the  parameter  recording 
program  operates  only  in  conjunction  with 
programmed  missions.  Ten  parameters  can  be 
monitored,  and  the  results,  in  continuous  trace 
form,  are  displayed  in  sets  of  two  parameters 
each.  There  is  no  provision  for  specifying 
tolerances  since  the  programmed  missions  contain 
the  tolerances.  Thus  the  parameter  recording 


PARAMETER  RECORDING  D  9 


PARAMETER 

CODE 

REF  VAL(F) 

TOLERANCE (L) 

STANDBY 

RECORDING 

HEADING  (DEG) 

HD 

360.0 

10.0 

0 

0 

ALTITUDE  (FT) 

AL 

15000.0 

100.0 

0 

0 

AIRSPEED  ( KIAS ) 

AS 

320.0 

20.0 

-  — G 

0 

ANGLE  OF  ATTACK  (UNITS) 

AA 

17.0 

3.0 

0 

0 

ANGLE  OF  BANK  (DEG) 

AB 

0.0 

15.0 

0 

0 

ACCELERATION  (G) 

AC 

1.0 

2.0 

0 

0 

FLAPS  (DEG) 

FL 

0.0 

30.0 

0 

0 

LANDING  GEAR  (1-DOWN) 

LG 

1 

0 

0 

0 

SPEEDBRAKE  (l=OUT) 

SB 

1 

0 

0 

0 

RATE  OF  CLIMB 

RC 

0.0 

500.0 

0 

0 

RATE  OF  DESCENT 

RD 

0.0 

500.0 

0 

0 

RATE  OF  TURN  (DEG/SEC) 

RT 

0.0 

1.0 

0 

0 

PITCH  RATE  (DEG/SEC) 

PR 

0.0 

15.0 

0 

0 

ROLL  RATE  (DEG/SEC) 

RO 

0.0 

15.0 

0 

0 

YAW  RATE  (DEG/ SEC) 

YR 

0.0 

15.0 

0 

0 

GCA/CCA  ELEVATION  (FT) 

GE 

0.0 

50.0 

0 

0 

GCA/CCA  AZIMUTH  (DEG) 

GA 

0.0 

100.0 

0 

0 

GCA/CCA  CENTERLINE  (FT) 

GC 

0.0 

100.0 

0 

0 

RPM,  LEFT  {%) 

RL 

82.0 

5.0 

0 

0 

RPM ,  RIGHT  {%) 

RR 

82.0 

5.0 

0 

0 

PITCH  ANGLE  (DEG) 

PA 

0.0 

15.0 

0 

0 

MACH  ( IMN ) 

MN 

0.68 

0.05 

0 

0 

TAG  AN  BEARING  (DEG) 

TB 

360.0 

2.0 

0 

0 

DME  (NM) 

TD 

20.0 

2.0 

0 

0 

ACLS  ELEVATION  (FT) 

AE 

0.0 

20.0 

0 

0 

ACLS  AZIMUTH  (DEG) 

AZ 

0.0* 

2.0 

0 

0 

ACLS  CENTERLINE  (FT) 

AR 

0.0 

50.0 

0 

0 

ENABLE  RECORDING  0 

DISABLE  RECORDING 
CLEAR  HEADINGS 


Figure  5.  Parameter  Recording  Display  (Partial)  for  EA-6B  WST 
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computer-derived  analysis  of  the  bomb  runs  (only 
the  impact  point  is  calculated),  although  it  would 
seem  that  this  would  be  an  excellent  use  of  A PM. 

"LC.‘A9'Ajr_  .Combat 

The  role  of  APM  in  air-to-air  combat  simula¬ 
tion  is  basically  similar  to  that  in  weapon 
scoring.  The  computer  analyzes  the  geometry  of 
the  intercept ,  checks  the  student’s  swiLchology, 
assesses  the  position  of  the  steering  dot  at  the 
moment  of  firing,  and  makes  an  assumption  about 
the  P).-  (probability  of  kill)  of  the  weapon  used, 
and  from  these  calculations  determines  whether  a 
kill  was  achieved  or  not.  An  instructor  would  be 
able  to  make  the  same  evaluation  subjectively, 
but  more  slowly,  of  course,  and  always  with  the 
possibil  ity  of  error. 

The  displays  Tor  air-to-air  combat  can  range 
from  highly  sophisticated,  depicting  dynamically 
the  three-dimensional  relative  position  of  the 
two  maneuvering  aircraft,  to  relatively  simple, 
consisting  merely  of  a  map  display  showing 
"ownship"  and  the  hostiles.  In  any  case,  the 
display  will  show  the  results  of  the  engagement  ~~ 
in  the  form  of  a  kill  or  no-kill  statement, 
usually  accompanied  by  some  type  of  analysis. 

Prouedt  •  re_  _Mo  n_i_to  r  i_rvg 

Procedur e  monitoring  is  a  urogram  that 


enables  the  instructor  to  evaluate  how  well  a 
student  accomplishes  the  required  steps  in  a 
published  procedure,  such  as  the  interior  inspec¬ 
tion  checklist  (a  "normal"  procedure)  or  the 
engine  failure  emergency  procedure.  When  proce¬ 
dure  monitoring  is  specified,  the  contractor  Is 
usually  required  to  develop  l  very  extensive 
library  of  CRT  display^,  normally  one  page  per 
procedure.  In  some  specifications  two  procedures 
per  page  are  stipulated. 

A  procedure  display  typically  lists  the 
steps  in  the  required  sequential  order,  and 
indicates  the  order  of  actual  accomplishment, 
either  by  an  additional  column  of  numbers  or 
by  listing  the  numbers  across  the  bottom  (see 
Figure  8).  In  some  simulators  the  instructor  i$ 
able  to  control  when  the  procedure  is  "active" 
and  subject  to  monitoring,  and  their  displays 
show  the  elapsed  time  for  completing  the  proce¬ 
dure  . 

There  are  no  known  specifications  that  call 
for  an  overall  evaluation  or  score  of  a  student's 
accomplishment  of  the  normal  and  emergency  proce¬ 
dures  that  he  is  faced  with  during  an  exercise. 

This  would  be  within  the  capabilities  of  APM,  but 
the  algorithms  would  have  to  cope  with  the  fact 
that  in  many  procedures  there  are  steps,  such  as 
those  that  state  "visually  check"  or  "as  required", 
that  cannot  be  evaluated  by  the  computer.  Current¬ 
ly  the  displays  simply  ignore  whether  these  steps  - -  ■ 
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Figure  8,  Procedure  Monitoring  Display  for  A-6E  OFT . — 
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are  accomplished  or  not,  or  assume  that  they  are. 
The  instructor  is  expected  to  make  the  necessary 
allowances  in  his  mental  evaluation. 

Usually,  simulators  that  have  on-board 
instructor  stations  do  not  have  the  procedure- 
moiitoring  feature,  presumably  on  the  premise 
that  the  instructor  can  adequately  determine  by 
over- the-s‘'oulder  observation  whether  the  student 


either  from  a  graphic  scale  or  'rom  a  numerical 
readout  how  much  the  student  is  deviating  hor  - 
zontally  and  vertically.  The  instructor  can  then 
provide  the  simulated  GCA  operator's  instructions 
to  the  pilot.  More  advanced  simulators  auto¬ 
matically  display  the  instructions  on  the  page, 
so  the  instructor  only  needs  to  read  them.  Auto¬ 
matic  voice  instructions  are  also  available,  as 
on  the  F/A-18  OFT. 


Some  simulators  use  the  same  type  of  display 
for  ILS  (instrument  landing  system)  approaches, 
except  that  operator's  instructions,  of  course, 
are  not  included.  Alternatively,  a  repeater 
instrument  at  the  instructor  station  reproduces 
the  ILS  indications  in  the  cockpit  and  enables 
the  instructor  to  visually  evaluate  the  approach. 
L ke  GCA's,  no  specification  requires  a  computer- 
derived  evaluation  or  score. 

Time  Plots 

Some  simulators  have  a  graphic  display  that 
depicts  certain  parameters,  such  as  altitude  and 
airspeed,  plotted  gainst  a  time  base  (see 
Figure  10).  This  type  of  display  is  usually 
shown  in  an  area  of  recurring  flight  data,  sone- 
, times  called  the  "reserved  area"  or  the  area  for 
"supplemental  data". 
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Figure  10.  Time  Plot  Display  for  CH-53D  OFT 

The  purpose  of  this  display  is  obscure.  It 
provides  a  brief  graphic  record--the  period 
covered  is  usually  around  10  minutes--but  no 
provision  is  made  to  include  a  reference  value  or 
tolerance.  One  suspects  that  it  is  a  carry-over 
from  an  out-of-date  concept.  While  it  is 
computer-derived,  it  falls  far  short  of  being  an 
example  of  APM. 

Terminal  Conditions 


Simulators  for  Corrier-Dased  aircraft  some¬ 
times  have  a  display  or  pj  lotout  that  provides 
data  related  to  the  landing  on  carrier  deck. 
The  A-6E  Night  Carrier  Landing  Trainer  (*'rLT)  has 
devoted  half  of  its  Performance  Evaluation  Page  to 
a  Terminal  Conditions  Display  which  automatically 
records  nine  parameters,  such  as  the  wire  caught, 
hook-ramp  clearance,  lateral  distance  from  center- 
line,  etc.  (see  Figure  11).  The  instructor  is 


able  to  print  the  data  for  later  use  in  the 
post-mission  critique. 
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Figure  11.  Terminal  Conditions  Display 
for  A-6E  NCLT 

In  other  simulators  a  similar  display  is 
sometimes  provided  for  the  terminal  conditions 
pertaining  to  landing  on  a  runway.  In  the  F/A-18 
OFT  the  Landing  Data  Display,  which  is  not  auto¬ 
matic  but  must  be  called  up  by  the  instructor, 
has  a  dual-purpose  display,  half  of  which  is 
devoted  to  carrier-oriented  parameters  and  half 
to  runway.  The  parameters  are  mostly  the  same, 
except  that  those  for  a  runway  include  "distance 
from  threshold"  rather  than  "wire  caught"  and 
"hook-ramp  clearance". 

In  the  F/A-18  OFT,  the  display  lists  the 
optimum  values  for  the  carrier-landing  parameters. 
The  optimum  hook-ramp  clearance,  for  example, 
is  stated  as  14.1  feet.  After  a  student  has 
completed  a  landing,  the  instructor  can  compare 
tne  actual  values  with  the  optimum  ones. 

As  observed  previously,  it  appears  that  APM 
could  be  called  on  to  go  one  step  farther  and 
provide  a  qualitative  assessment  of  the  student's 
performance.  Since  the  optimum  values  are  known, 
all  that  remains  is  for  the  users  to  determine 
the  range  for  satisfactory  achievement.  The 
computer  would  do  the  rest. 

Ejection  Data 

A  display  similar  to  terminal  conditions 
displays  is  the  ejection  data  display,  which 
presents  to  the  instructor  the  essential  infor¬ 
mation  regarding  an  ejection,  if  the  student 
simulates  one.  In  the  f/A-18  OFT  the  display 
reports  the  indicated  airspeed,  altitude, 
vertical  velocity,  bank  angle,  and  pitch  angle-- 
all  of  which  are  relevant  to  the  success  cf  the 
ejection--and  then,  in  a  succint  demonstration 
of  APM,  concludes  by  answering  with  yes  or  no 
the  question  "Successful?" 

Event  Pi i ^*out 


Another  feature  r°lated  to  performance 


measurement  but  with  an  obscure  purpose  is  that 
referred  to  in  specifications  as  "event  printout". 
In  the  A-6E  OFT,  for  example,  this  program  auto¬ 
matically  prints  out  the  time  of  occurrence  of 
18  flight-related  events.  The  events  include 
takeoffs,  gear  up,  flaps/slats  up,  stores  release, 
master  caution  light  illuminated,  land^ig  touch¬ 
down,  and  others.  For  some  events,  the  associated 
airspeed,  altitude,  and  angle  of  attack  are  also 
printed  out.  The  printout  is  made  eithe-'  on  the 
parameter  recording  sheet,  in  the  Remarks  column, 
or  separately  if  parameter  recording  isnot  in  use. 

Presumably  the  instructor  will  use  the 
printout  as  an  aid  in  critiquing  or  evaluating 
the  student.  If  the  event  printout  program  is 
used  in  conjunction  with  parameter  recording  it 
can  assist  the  instructor  to  reconstruct  tne 
nrssion  and  interpret  the  strip  charts.  Its  use¬ 
fulness  alone  is  questionable. 

USER  OPINIONS 

Representative  user  opinions  about  APM  are 
difficult  to  obtain.  Instructors  who  regularly 
man  the  instructor  stations  are  the  most  knowl¬ 
edgeable  persons  about  the  usefulness  of  the 
different  variations,  but  a  consensus  can  be 
obtained  only  by  questionnaires  or  personal 
interviews--an  impracticable  approach  in  view 
of  the  widely  scattered  locations  of  most 
simulators.  The  two  existing  A-6E  WSTs  .re 
located  as  far  apart  as  NAS  Whidbey  Island, 
Washington,  and  NAS  Oceana,  Virginia.  The  A- 1 0 
OFTs  that  have  been  the  longest  operational  are 
located  at  Davis  Monthan  AFB,  Arizona,  and 
Myrtle  Beach  AFB,  South  Carolina.  The  EA-6B  WST 
is  at  NAS  Whidbey  Island  and  the  A-6E  OFT  is  at 
MCmS  Cherry  Point,  North.  Carolina.  A  comprehen¬ 
sive  survey  covering  such  locations  would  be  very 
costly  and  time-consuming. 

Another  problem  is  the  fact  that  the  newest 
simulators  have  the  most  complete  suite  of  per¬ 
formance  measuring  features,  but  some  of  these 
simulators  are  too  new  for  a  definitive  assess¬ 
ment.  The  F - 1 6  OFT,  the  A-6E  OFT,  and  the  F/A-iS 
OFT  are  very  current  simulators  with  many  examples 
of  APM;  unfortunate-^  none  has  been  operational 
tong  enough  for  the  users  to  have  acquired  a  body 
of  opinion. 

In  the  absence  of  a  thorough  survey  of 
instructor's  views,  the  observations  and  general 
opinions  of  a  variety  of  user  representatives 
have  bee-  obtained,  mostly  by  telephone.  Tnese 
persors  include  instructors  and  device  operators, 
supervisors  of  training  at  the  base/station  level, 
and  staff  officers  at  the  headquarters  level  who 
are  responsible  for  planning  simulator  utilization. 
They  all  have  considerable  experience  in  flight 
training  and  the  use  of  simulators,  and  they  have 
varying  degrees  of  knowledge  about  tl  types  of 
APM  available. 

Checkrides 

From  the  available  opinions  it  must  con¬ 
cluded  that  checkrides  are  used  very  little,  even 
with  the  simulators  that  have  the  most  advanced 
capability.  The  general  range  of  use,  among  the 


different  simulators  investigated,  is  from  zero 
to  very  infrequently. 

The  reasons  for  non-use  are  complex.  In  one 
case,  only  one  checkride  was  delivered,  of  ten 
originally  required  by  the  contract;  the 
remaining  nine  were  omitted  in  return  for  other 
"considerations".  The  user  was  instructed  in 
how  to  develop  his  own,  but  to  date  he  has  not 
done  so.  To  compound  the  problem,  the  one 
checkride  delivered  was  more  of  a  demonstration 
of  the  capability  than  a  mission  tailored  to  the 
user's  needs. 

In  another  case,  difficulty  in  operating 
the  program  has  been  the  principal  reason  for 
lack  of  use.  All  of  the  leg  ends  are  defined 
by  latitude  and  longitude,  so  if  the  student 
does  not  come  acceptably  close  to  the  specified 
"'ocation,  the  instructor  must  intervene,  which 
is  a  frequent  occurrence.  In  this  simulator 
he  slews  the  aircraft  so  as  to  achieve  the  leg 
end.  This  process  not  only  is  cumbersome  but 
also  it  invalidates  the  scoring. 

It  can  be  predicted  that  any  checkride 
program  that  is  difficult  to  update  will  tend 
toward  disuse.  Usually  specifications  require 
that  program  updates,  including  changes  in 
display  pages,  be  able  to  be  accomplished  via 
Plan  Mode  (a  term  used  in  U.S.  Navy  specifica¬ 
tions).  Ideally,  Plan  Mode  should  be  easily 
operated  by  instructors  or  maintenance  personnel  . 
If  it  is  not  (often  only  because  the  instructions 
are  unclear),  the  checkrides  will  become  static, 
out-of-date,  and  unused. 

Underlying  all  other  reasons  for  checkrides 
not  being  used  is  the  probability  that  instruc¬ 
tors  subconsciously  resist  computer  evaluation 
of  a  student.  Why  this  attitude  should  exist-- 
whether  it  is  based  on  ignorance,  prejudice,  or 
parochial ism--is  not  within  the  scope  of  this 
paper.  But  to  some  degree  it  exists.  Possibly 
a  contributing  factor  is  the  feeling,  at  the 
base  or  station  level,  that  the  requirement  for 
checkrioes  in  the  specification  originated  else¬ 
where  . 

Automi ssions 

Relatively  few  operational  simulator1"  have 
automi ssions ,  therefore  the  user  opinion  of  this 
feature  is  limited.  The  available  evidence  is 
that  acceptance  and  employment  of  automissions 
will  be  similar  to  those  of  checkrides,  although 
possibly  there  will  be  better  utilization  of 
automissions  because  the  instructor  is  more  in 
control,  particularly  through  his  ability  to 
vary  the  difficulty  of  the  mission  based  on  his 
judgment  of  the  proficiency  of  the  student. 

Mon  i  to  red  Maneuv  c-  r  s 

Since  tiie  moni tored-maneuver  approach 
originated  at  the  U.S.  Coast  Guard  Aviation 
Training  Center,  at  Mobile,  Alabama,  where  the 
simulator  involved  will  be  located,  it  can  be 
expected  that  the  users  will  be  reasonably 
pleased  with  it.  The  simulators  are  over  a  year 
f-'Oiii  delivery,  however,  so  how  much  it  will 
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actually  be  used  is  to  be  determined;  but  it  is 
anticipated  that  this  form  of  APM  will  be  an 
integral  part  of  the  Coast  Guard's  training 
syl  labus . 

Semi -Automated  Performance  Measurement 

No  information  is  available  on  how  much 
semi -automated  performance  measurement  is  used 
in  the  A-4H  and  N  simulators.  In  view  of  the 
demands  that  this  program  makes  on  operators,  it 
it  unlikely  that  it  is  used  much. 

Parameter  Recordi ng 

Parameter  recording  is  another  example  of  APM 
that  is  used  either  not  at  all  or  very  little. 

The  reasons  for  non-use  are  more  apparent  than  for 
checkr ides--parameter  recording  is  difficult  to 
operate,  unless  the  number  of  parameters  monitored 
is  kept  to  a  minimum;  and  the  results  require 
analysis  and  interpretation  by  the  instructor, 
unless  the  maneuver  is  quite  simple.  Parameter 
recording  could  be  used,  for  example,  to  monitor 
the  g-schedule  during  a  loop,  or  to  monitor  the 
rate  of  turn  during  a  TACAN  arc.  It  could  not  be 
used  easily  during  a  series  of  acrobatic  maneuvers 
or  during  an  entire  instrument  approach.  The 
statement  was  made  by  an  interviewee  that  parameter 
recording  is  more  applicable  to  basic  f 1 y i ng 
training  than  to  advanced  or  tactical  training. 

That  may  be  an  accurate  observation. 

In  one  simulator,  the  A-6E  Night  Carrier 
Landing  Trainer  (NCLT),  it  has  been  reported  that 
parameter  recording  is  used  extensively.  The 
mission  of  the  NCLT  is  to  enable  a  student  co  make 
repeated  carrier  approaches  and  landings.  The 
parameters  to  be  monitored  can  be  preplanned  on  a 
CRT  page  and  entered  simultaneously  at  the  begin¬ 
ning  of  each  approach,  and  the  results  are 
displayed  and  later  printed  on  a  two-minute  strip 
recording.  Thus,  the  mission  of  the  trainer  is 
simple  and  its  parameter  recording  capability  is 
compatible.  An  additional  feature  that  may 
influence  the  user's  opinion  is  the  ability  of 
the  instructor,  at  the  completion  of  each  landing, 
to  display  the  strip  recordings  to  the  student  via 
the  visual  system  at  the  trainee  station,  i.e., 
in  the  cockpit. 

0 t her  For ms_o f_ APM 

The  other  forms  of  APM  discussed  previously-- 
weapon  scoring,  procedure  monitoring,  GCA/ILS 
monitoring,  etc. --are  generally  simpler,  easier 
to  operate,  and  easier  to  interpret  than  the  full- 
mission  programs  such  as  cneckrides,  automissions, 
and  parameter  recording;  and  they  are  well 
accepted  and  extensively  used,  with  some  excep¬ 
tions  . 

Weapon  scoring  is  used  because  it  provides 
an  assessment,  usually  graphically,  that  the 
instructor  is  unable  to  make  himself.  Likewise, 
procedure  monitoring  is  used  at  a  remote  instruc¬ 
tor  station  because  the  instructor  has  no  other 
way  to  determine  what  the  student  is  doing  in  the 
cockpit.  GCA/ILS  monitoring  is  accomplished 
simply  and  graphically,  and  consequently  is  well 
accepted.  Time  plot  displays,  as  described  in  the 
foregoing,  are  usually  contained  in  the  reserved 


area  and  hence  are  always  present,  but  if  they 
were  assigned  to  a  separate  display,  it  is 
likely  that  they  would  never  be  used.  Terminal 
conditions  displays  are  used  when  they  are 
generated  automatically;  when  they  are  not  they 
are  often  overlooxed.  The  use  of  event  printout 
is  usually  related  to  the  use  of  parameter 
recording,  thus  it  suffers  from  the  same  problems. 

CONCLUSIONS 

Many  conclusions  can  be  drawn  from  a  study 
of  APM.  The  most  obvious  one  is  that  the 
Government  spends  a  great  deal  of  money  procuring 
programs  that  are  used  very  little.  However,  it 
does  not  follow  that  less  emphasis  should  be 
placed  on  APM.  There  are  many  ways  in  which  APM 
can  assist  and  improve  training;  the  direction  of 
effort  should  be  to  in  ure  that  the  money  is 
spent  effectively. 

It  appears  from  the  language  in  specifica¬ 
tions  tha'  the  Government  is  unsure  about  what 
it  wants.  In  many  cases  the  description  of 
required  capabilities  is  unclear.  "Boilerplate" 
language  is  used  that  can  be  traced  back  to 
specifications  many  years  old;  the  meaning  was 
vague  then  and  it  still  is.  Furthermore,  incon¬ 
sistencies  and  redundancies  exist  within  the 
same  specification. 

As  a  first  step  toward  being  more  precise 
about  its  needs,  the  Government  should  standardize 
its  terminology  and  publish  a  glossary  of 
simulator-related  definitions  for  use  in  all 
specifications.  The  term  "automated  training 
exercise",  for  example,  should  have  an  exact 
meaning,  or  if  "automi ssior."  is  used  instead, 
it  should  mean  the  same  to  the  Air  Force  as  to 
the  Navy.  Or,  alternatively,  a  more  descriptive 
term  should  be  devised  and  made  standard. 

The  subject  of  standardization  of  "instruc¬ 
tional  feature  requirements",  which  includes  many 
other  features  besides  APM,  has  been  addressed  by 
Pohlmann,  Isley,  and  Caro,  and  the  development 
of  "design  guides"  has  been  proposed.^)  Some 
recent  specifications  contain  design  guides, 
although  they  do  not  cover  APM  features  as 
thoroughly  as  might  be  desired.  The  concept  of 
design  guides  is  subject  to  debate--if  it  is  too 
rigid  it  could  discourage  initiative  and  innova- 
tion--hut  at  least  it  would  provide  a  base  of  a 
common,  understandable  terminology. 

As  the  second  step  in  defining  its  needs, 
the  Government  should  conduct  an  organized  study, 
or  a  series  of  studies,  of  the  uses  of  APM.  The 
principal  purpose  of  such  research  would  be  to 
determine  which  of  the  different  variations  of 
APM  are  the  most  useful,  .'he  questions  to  be 
answered  are  simple:  Should  checkrides  and 
parameter  recording  both  be  procured  on  ar.  OFT? 

Are  automissions  cost-effective?  Should  the  use 
of  tolerances  be  abandoned  in  favor  of  more 
sophisticated,  comparative  scores?  Should  proce¬ 
dure  monitoring  be  used  with  on-board  instructor 
stations?  How  many  of  the  marginal  versions  of 
APM--terminal  conditions  displays,  time  plots, 
event  printout--shoul d  be  procured? 

The  fact  that  a  feature  is  little  used  is 
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not  proof  that  it  has  little  potential  value.  In 
many  cases,  the  use  of  instructional  features 
depends  on  the  training  and  motivation  of  the 
instructors.  The  fact  thit  instructors  frequently 
do  not  know  how  or  when  to  use  "advanced  simulator 
training  features"  has  been  commented  on  by  Caro 
in  a  report  entitled  Some  Current  Problems  in 
Siinul ator  Design,  Testing,  and  Usef6T!  Pro ba bl y 
their  ignorance  or  disinterest  is  greater  in  the 
area  of  APM  than  in  any  other  general  feature. 

Perhaps  what  is  needed  more  than  any  single 
action  is  better  communication  and  coordination 
among  the  procuring  agencies  and  users.  Too 
often  the  mistakes  made  with  one  simulator  are 
repeated  in  others,  through  specifications  that, 
in  the  field  of  APM,  have  simply  been  copied  from 
preceding  ones.  Even  if  serious  mistakes  have 
not  been  made,  there  are  lessons  learned  in  every 
design  that  are  not  being  passed  on  -j  succeeding 
ones . 

A  fall-out  from  better  communication  would 
be  more  appreciation  at  the  basc/station  level 
of  the  ways  in  which  APM  can  assist  instructors. 
During  training  sessions  there  are  many  demands 
on  simulator  instructors;  they  would  benefit  from 
the  computer's  assistance  if  they  better  under¬ 
stood  its  capabilities. 
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ABSTRACT 
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With  the  widespread  availability  cf  word  processing  within  businesses,  the 
governnent  and  the  homes  cf  many  Engineers,  it  has  become  increasingly  apparent 
that  this  tool  could  be  used  effectively  to  reduce  the  high  costs  of  contractual 
data  preparation  and  review  while  simultaneously  improving  the  quality  of  this 
data.  This  paper  addresses  the  use  of  word  processing  on  an  NTEC  program.  The 
primary  objective  is  to  reduce  the  total  cost  and  elapsed  time  of  the 
preparation- through-acceptance  (PTA)  cycle  of  the  contractual  documents. 


INTRODUCTION 

During  the  early  stages  of  an  NTEC 
study  progam  (in  which  deliverable 
documentation  consisted  primarily  of 
reports  and  tabular  data)  the  use  of  a 
microcomputer  based  word  processing  system 
was  considered  as  a  cost  effective 
approach  that  would  entail  minimum  cost 
a. id  schedule  risk. 

The  approach  was  considered 
appropriate  due  to  the  extensive  review 
comments  anticipated  as  well  us  the 
significant  updating  of  the  data  that 
would  occur  over  the  life  of  the  program. 
Although  block  and  functional  flow 
diagrams  were  also  developed  during  the 
study  they  constituted  a  relatively  small 
percentage  of  the  total  data  package. 

Since  normal  trainer  developments 
also  share  some  of  these  characteristics, 
it  was  evident  that  the  results  of  this 
effort  could  be  applied  to  these  trainer 
programs  as  well. 

Our  primary  hypothesis  is  that  much 
of  the  data  development  time  could  be 
reduced  if  the  PTA  cycle  could  be  reduced. 
Even  though  most  of  the  data  items 
prepared  under  a  typical  trainer  contract 
are  Category  II  or  III  (data  items  that 
would  normally  te  prepared  by  the 
contractor  and  require  only  reformatting 
cr  can  be  delivered  outright),  the 
practical  aspects  are  that  even  if  the 
data  is  prepared  for  use  by  the 
contractor,  its  submission  and  subsequent 
rework  for  approval  requires  a 
significant  expenditure  of  effort  due  to 
the  length  of  time  (and  hence  cost) 
between  generation  and  approval. 
Shortening  this  cycle  would  mane  the  data 
more  meaningful  to  both  the  contractor  and 
NTEC  and  hence  reduce  the  total  cost. 


DATA  PREPARATION'1  AND  REVIEW 

Before  discussing  the  results  of 
this  approach  it  is  necessary  to  examine 
the  data  PTA  cycle  to  gain  a  better 
understanding  of  what  is  to  be 
accomplished.  Figure  'll  shows  this  typical 
cycle.  Both  the  time  sbans  and  the  actual 
"roadmap"  will  vary  Ifrom  item- to-item , 
con tract-to-con tract  and 

contractor-to-contructor ,  however,  the 

essential  processes  do  not  vary  widely. 
Note  that  over  305&  ,of  the  elapsed  time 
between  the  completion  iof  the  rough  draft 
write-up  and  receipt  of  the  NTEC  comments 
is  in  distribution  and  mailing.  Only 

about  25%  of  the  tine  is  spent  with  the 

document  in  the  nanis  of  either  the 
originator  or  the  NTEC  recipient. 

The  approach  to  be  described 

concentrates  on  the  following  specific 
areas : 

1.  Initial  data  generation 

2.  Government  review  and  submission 
of  comments  to  originator 

3.  Incorporation  cf  review  comments 

These  areas  indirectly  or  directly 
account  for  the  majority  of  the  time  spent 
in  the  PTA  cycle.  Not  specifically 
addressed  were  the  areas  of. 

1.  Contractor  design/document  review 
procedures 

2.  NTEC  review-procedures 

3.  Formal  Contractor  cr  Government 
handling  procedures 

Through  the  use  of  word  processing 
and  data  transmission  equipment 

significant  improvement  in  the  PTA  cycle 
time  was  achieved  without  disrupting 
organizational  procedures. 
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Some  processes  were  changed  and/or 
expanded  in  order  to  accommodate  the 
handling  of  computer  diskettes.  While 
basic  generation  and  review  procedures 
were  left  unchanged,  a  procedure  was 
developed  to  contractually  protect  both 
the  contractor  and  NTEC  when  data  was 
being  developed  or  passed  on-line  from 
computer  to  computer. 


Initial  Data  Generation 

To  assist  the  originator  in 
developing  the  data,  a  commercially 
available  microcomputer,  word  processing 
software,  printer  and  telephone  modem  were 
purchased  under  contract.  Training  of  the 
originator’s  secretary  enabled  her  to 
support  the  initial  data  preparation  and 
to  reduce  the  dependency  on  the  direct 
charge  (and  fairly  inflexible)  typing 
pool.  The  modem  was  provided  so  that  data 
prepared  on  the  microcomputer  could  be 
passed  directly  to/from  the  word 
processing  equipment  used  by  the  typing 
pool.  Some  difficulties  were  encountered 
in  attempting  to  complete  this  interface 
with  the  typing  pool  equipment.  As  an 
interim  measure,  files  were  established  on 
a  remote  time  share  computer  system.  These 
files  were  read  from,  or  written  into,  by 
both  the  microcomputers  and  the  typing 
pool  equipment.  (A  vexing  problem 
developed  in  the  way  in  which  various 
computers  and  word  processing  software 
treat  ’’carriage  retc.rn/line  feeds".  This 
is  discussed  under  lessons  learned.)  At 
the  time  of  this  writing,  methods  were 
being  evaluated  for  upgrading  this 
communications  method. 

A  significant  portion  of  one  of  the 
deliverable  documents  (some  200  pages  of 
tabular  data)  had  previously  been  prepared 
on  the  remote  time-share  computer.  This 
data  was  down-loaded  to  the  microcomputer. 
The  originator  was  then  able  to  edit  the 
data  file  directly  and  incorporate  it  into 
the  document.  Had  this  capability  not 
existed,  the  entire  document  would  have 
had  to  be  essentially  re-generated  (the 
edits  performed  were  substantial  and  not 
readily  describable  to  a  typist).  By 
using  this  previously  generated  data  file, 
the  originator  and  editor  had  to 
concentrate  only  on  the  changes,  not  the 
entire  document.  The  originator  completely 
revised  these  tables  in  about  two  days 
(The  original  typing,  editing  and 
reviewing  took  more  than  two  weeks.). 
Since  many  deliverable  data  items  consist 
of  portions  of  previously  developed  data, 
this  extraction  and  edit  process  can  be 
used  effectively  to  reduce  the  time  and 
cost  of  developing  other  data  items. 

Due  to  the  problems  encountered  in 
directly  transmitting  tc/from  the 
contractor's  word-processing  equipment, 
only  the  microcomputers  were  used  for 
developing  the  data  delivered  under  the 
contract.  Accordingly,  some  format 
requirements  were  relaxed,  or  waived,  in 


order  to  accommodate  the  orocess  (the 
primary  limitations  were  split  page 
format,  relocating  artwork,  and  the  use  of 
underlining  and  change  bars.) 

The  general  process  used  by  the 
originator  was  to  have  the  document  typed 
by  his  secretary  using  the  microcomputer 
and  wojrd  processing  software  and  then 
mark  up  a  printed  copy  of  the  rough  draft 
for  editing  on  the  microcomputer.  Editing 
was  performed  either  by  the  originator’s 
secretary,  the  editor  or  by  the 
originator.  The  data  was  stored  on  5-1/4 
inch  floppy  diskettes  which  were  formally 
"cycled”.  (Diskette  cycling  is  discussed 
in  Appendix  A.) 

In  accordance  with  the  contractor’s 
Engineering  procedures,  a  design  review 
was  conducted  using  printout's  of  the  data 
package.  The  majority  of  the  design 
review  team  generated  comments  in  a  normal 
manner:  i.e.  FROM/TO  changes.  ~ne  team 
member  incorporated  his  comments  directly 
onto  a  copy  of  the  original  diskette.  His 
changes  were  indicated  by  underlining 
changes  and  additions.  Blocks  of  deleted 
data  were  indicated  by  the  string 
’’••deleted** ” .  Individual  words  were 
merely  deleted.  After  review  of  his 
comments,  his  diskette  was  copied  and  the 
deleted  phrase  removed  with  the 
microcomputer's  editor.  This  process  saved 
the  reviewer  a  significant  amount  of  time 
and  effort  in  writing  his  comments.  It 
also  saved  the  originator  time 
incorporating  them.  This  process  did, 
however,  require  the  reviewer  to  have 
access  to  a  compatible  computer  system  and 
know  how  to  use  it. 

In  another  case,  the  reviewer 
transmitted  the  data  to  the  remote 
time-share  computer,  and  then  incorporated 
his  comments  on  that  data  file.  This  file 
was  transmitted  back  to  the  microcomputer 
for  incorporation  into  tne  complete 
document.  This  latter  process  was  used 
once  when  the  microcompute''*  was  in  use  for 
long  periods  of  time  and  also  when  the 
reviewer  was  out  of  town  but  had  access  to 
a  data  terminal  and  telephone. 

Government  Review  And  Submission  Of 
Comments  To  Originator 

By  far  the  most  beneficial  aspects 
of  this  approach  are  in  the  Government 
(NTEC)  review,  contractor  update  and 
resubmission  process. 

NTEC’s  Project  Engineer  was  provided 
a  microcomputer,  printer  telephone  modem 
and  word-processing  software  identical  to 
that  of  the  contractor.  To  expedite 
computer-to-computer  communications 
between  the  contractor  and  NTEC,  the 
remote  time  share  computer  was  used  as  a 
24  hour  intermediary  to  which  data  files 
were  sent  and  retreived.  Th’ s  process  was 
used  to  allow  communication  "ia  TYMNET 
which  has  "cleaner"  data  lines  than 
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In  some  cases,  data  was  submitted 
informally  for  review,  updated  and  then 
formally  submitted.  In  other  cas^s  the 
data  was  prepared  and  submitted  formally, 
without  an  informal  review.  In  both 
cases,  diskettes  and  a  printout  were 
included  in  the  submission,  and  in  both 
cases  NTEC  comments  were  made  directly  on 
the  diskette  by  underlining  changes  and 
additions  and,  by  marking  deleted  blocks 
cf  data  by  the  string  "**  deleted  **". 
•This  is  the  same  process  used  by  the 
originator).  A  copy  of  the  updated 
diskette  was  returned  to  the  contractor 
rcr  formal  re-submission.  In  some 
instances,  advance  comments  were  also 
transmitted  back  to  the  originator  via  the 
remote  time-share  computer.  By  providing 
equipment  on  which  the  NTEC  reviewer  can 
make  changes  directly  on  the  submitted 
diskettes,  a  significant  saving  in  time 
and  cost  for  both  the  Contractor  and  NTEC 
was  achieved. 

Not  all  comments  were  incorporated 
by  the  reviewer.  Some  required  additional 
cat a  and/or  restructuring  by  the 
contractor.  However,  the  ease  of 
incorporating  this  data  greatly  enhanced 
the  process.  (As  with  the  Contractor's 
review,  the  data  was  never  completely 
retyped,  therefore  only  the  areas  cf 
change  required  an  extensive  review.) 

There  were  instances  when  the 
contractor  was  meeting  at  NTEC  and 
incorporated  on-the-spot  changes  or 
additions  to  the  data  package  by  locally 
updating  a  diskette.  The  data  was  then 
sent  over  the  telephone  to  U e 
contractor's  microcomputer  for  formal 
resubmission  -  all  within  a  matter  of 
hours . 

This  process  virtually  eliminated 
the  reed  for  "FROM-TO"  comments  and 
enabled  NTEC's  Project  Engineer  to  bj  mere 
specific  in  his  comments. 


Incorporation  Of  Peview  Comments 

Many  cf  the  aspects  of  incorporating 
the  reviewer's  comments  were  discussed 
previously.  However,  it  is  significant, 
that  it  was  never  necessary  to  retype 
major  portions  of  the  document  and  the 
reviewers  comments  were  usually  clear  ard 
unambiguous  since  most  cf  them  were 
incorporated  directly. 

Ir.  some  ir. stances  the  data  diskettes, 
were  transmitted  back  to  the  remote 
time-share  compute*  for  integration.  Ir 
most  cases,  however,  the  crieir.atcr  ard 
his  secretary  made  all  of  the  charges  ret 
already  incorporated  by  the  reviewer. 


Although  a  direct  diskette  to 
diskette  submission  of  data  requires 
hardware  and  software  compatibility,  the 
use  of  an  intermediate,  remote,  time-share 
computer,  or  standard  RS232  communications 
link  could  be  used  for  the  transmission  of 
data  for  review  purposes.  Imbedded 
.-•rinting  and  format  commands  are  usually 
not  compatible  between  different  word 
processing  software,  therefore,  the  review 
document  data  would  have  to  be  sent 
without  these  commands.  Where 
incompatible  hardware/software  exist,  th13 
diskette  would  not  be  submitted  with  the 
data  package. 


EVALUATION 

Since  the  documents  had  not  been 
developed  using  both  this  approach  and  a 
more  conventional  process,  an  auditable 
FROK-TC  comparison  cannot  be  made. 
However,  based  upon  the  experience  of  ‘he 
Contractor,  time  estimates  have  been 
prepared  for  three  of  the  data  items. 
Table  I  shows  these  results.  At  the 
time  cf  this  writing,  many  start-up 
problems  had  been  coped  with  and  growing 
pains  felt,  A  cross-section  of  technical 
and  administrative  documents  had  not  been 
submitted  from  which  to  draw  financially 
auditable  conclusions  on  the  general  use 
of  this  approach.  There  is  no  doubt  that 
there  will  be  a  small  number  of  documents 
ir.  which  either  the  content,  format  or  the 
method  of  generation  is  incompatible  with 
this  approach. 

TAPLE  I  COMPARATIVE  TIME 


CDRI.  ITEM 

PAGE  COUNTS 

TIME 

(HOURS) 

Text 

Tables 

list. 

Est . 

Actual 

1  31 

4 9 

6 

202 

1  47 

2  2 

77 

29 

284 

135 

3  201 

117 

57 

1000 

360 

Both  NTEC  ar.d  the  contractor  are 
sufficiently  confident  of  this  approach 
that  the  majority  cf  the  data  items  on 
se\eral  contracts  will  be  prepared  and 
processed  using  many  cr  all  of  these 
techniques . 


LESSONS  LEARNED  AND  PROBLEM  AFEAS 

Although  this  approach  has  proved  to 
te  cost  effective,  there  are  several  areas 
which  must  be  considered: 

1.  Since  this  process  relies 
heavily  on  direct  engir.ecr-to-cngir.eer 
communications,  document  control 
procedures  are  necessary  to  protect,  both 
the  Contractor  and  NTEC.  A  preliminary 
set  cf  procedures  has  beer  developed  for 
use  by  the  Contractor  to  ensure  that 
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control  is  maintained  over  the  data 
submitted  and  that  committments  are  not 
made  unofficially.  A  summary  of  these 
procedures  is  contained  in  Appendix  A. 

2.  At  the  time  of  this  writing,  the 
most  appropriate  documents  to  address  in 
this  mannei  appear  to  be: 

a.  Correspondence 

b.  Statements  of  Work 

c.  Specifications 

d.  Status  and  Progress  reports 

e.  Technical  reports  (those  with 

minimum  artwork  and  a 

minimum  of  mathematical 
equations ) 

Investigations  are  currently 
underway  to  determine  how  much  of  the 
typical  ILS  data  can  be  effectively 
hardled  with  this  approach. 

3.  Many  remote,  time-share  computer 
systems  may  rot  accept  a  "string  input 
mode"  but  are  limited  to  lines  of  less 
than  250  characters.  Since  many  word 
processing  systems  are  not  li ne-cr iented , 
they  may  not  be  compatible.  This  problem 
has  beer  experienced  and  at  the  time  of 
this  writing  is  under  irvestigat ior . 

Micro-computers  have  limited 
memory  and  somewhat  limited  word 
processing  capabilities.  The  most 
effective  use  of  them  is  in  the 
preparation  and  review  of  data  which  can 
be  broken  down  into  15000  to  20000 
character  segments.  Although  it  is  more 
effective  to  use  larger  word  processors 
for  the  final  output,  microcomputers  offer 
the  advantage  of  being  more  "user 
friendly".  This  capability  is  valuable  ir. 
the  preparation  of  technical  data  which 
may  require  computational  ability  and/or 
data  base  manipulation.  In  addition,  it 
is  typically  more  cost  effective  to 
procure  computers  than  word  processors  for 
Engineering  groups. 

5.  Most  of  this  process  car  be 
accomplished  through  the  use  cf  remote 
time-share  computer  systems  exclusively. 
Wt  have  found  that  reliability  is  higher, 
speed  is  increased  and  total  costs  are 
less  using  local  microcomputers  or  local 
multi-terminal  processors.  In  any  case, 
the  terminal  at  which  the  operator  is 
located  should  be  "smart",  have  at  least 
2K  to  4K  of  local  character  memory  and 
support  diskettes.  T ne  system  currently 
empicyed  is  shown  ir  Figure  2, 

6.  Direct.  computer-tc-computer 

communications  using  conventional  voice 
grade  telephone  lines  has  not  been  found 
to  be  satisfactory  at  speeds  over  30-60 
baud.  Communications  accessed  via  TYMNET 
are  satisfactory  to  300  baud  over  a 
conventional  voice  grade  .line.  A  local 
call  over  a  data  line  is  satisfactory  to 
1200  band.  On  some  computer  systems  the 
CHT  refresh  time  may  be  sufficiently  long 
that  some  data  is  missed  if  transmissions 


ever  600  baud  are  attempted.  Selection  of 
the  communications  software  should  take 
this  into  account.  The  software  should 
provide  a  "PAGE"  vis-a-vis  "SCROLL"  mode 
of  operation  to  get  around  this  problem. 

7.  Time-share  computer  systems 
typically  provide  24  hour  backup.  This 
protects  the  user  from  losing  more  than  24 
hours  cf  data.  Micro-computers  do  not 
offer  such  protection,  therefore  it  is 
mandatory  that  diskette  cycling  be  used. 
This  cycling  procedure,  which  is  discussed 
ir.  Appendix  A,  involves  maintaining  at 
least  three  copies  of  each  diskette,  the 
most  recent  version  of  which  is  copied 
onto  the  oldest  version. 

8.  Format  requirements  found  in 
many  Data  Item  Descriptions  (DID's)  are 
r.ot  generally  compatible  with  this 
approach.  The  emphasis  must  be  placed  on 
legibility  and  accuracy  rather  than 
format.  The  primary  problem  areas  are: 

a.  Split-page  format 

b  Eoldface  and  underlining 

c.  Location  of  ncn-textual  data  such 

3  S 

artwork,  photographs,  etc. 

d.  Character  font 

e.  Tabular  formats 

The  more  sophisticated  word 
processing  systems  can  usually 
accrmmcdate  these  requirements, 
however  many  cannot  interface 
with  any  ether  system  and  many 
require  that  format,  commands  be 
permanently  imbedded  ir,  the  text. 
These  characteristics 

significantly  reduce  most  cl'  the 
benefits  derived  from  this 
approach.  Whether  these 

characteristics  are  "MUSTS"  or 
"NICE  TO  HAVE’S"  must  be 
seriously  reviewed  by  both  the 
contractor  and  NTEC. 


SUMMARY 

The  use  of  this  approach  for 
developing  deliverable  documentation  shows 
considerable  promise  in  significantly 
reducing  both  the  time  and  cost  cf  data 
preparat i on . 

Although  the  project  is  just 
underway,  most  of  the  objectives  are  being 
achieved.  The  status  at  the  time  of  this 
writing  shows: 

1  .  A  505  -  751  reduction  ir. 
turnaround  time  is  being  realized 
primarily  due  to  the  direct  incorpcrat ion 
of  comments,  versus  the  conventional 
"TC-FROM"  mechanism. 

2.  The  documents  show  a  significant 
increase  in  accuracy  due  to  a  reduction  in 
Lhe  number  of  "rc-types"  and  a  significant 
decrease  in  the  preparation  lead  time.  In 
seme  instances,  the  formal  submissions 
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contained  data  only  a  few  hours  old. 

3.  Control  and  handling  procedures 
were  developed  to  ensure  that  the 
potential  informality  did  not  compromise 
NTEC  or  the  contractor. 

4.  Format  guidelines  were  relaxed 
with  emphasis  being  placed  on  legibility 
and  technical  content. 
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1  i-  r  a  t  "  p n  g  e  "  d  a  t «  ,  d  i  s k  <■  ?  t  »■ 
c! a r- a  •  Ensure  accur a c y  c r  <.  o 
diskette  first  "page".  Er,r;u 
diskettes  are  read lble  (i 
LIS’  on  the  computer).  T 
ftttu  ined  by  Date  Me.  nag  erne  ri 
tt.e  masters  and  must  be 
such.  The  printed  copy,  not 
shall  be  routed  for  signatur 


ntroi  data  hr  C 
re  that  backup 
.e,f  perf errfi  a 
he  diskettes 
a  r **  r-r,nft  i  d#*red 
controlled  as 
the  diskette, 


>■' 1 1  h  the  advent  of  microcomputers 
and  word  processing,  documents  may  he 
uejiverod  on  diskette  and/or  transmitted 
electronically  directly  to  the  receiver. 

procedure  defines  the  process  for 
\  -rcpu  ter  -  computer  rommeru  cations  and 
delivery  of  documentation  on  diskette 
between  the  customer  and  the  Contractor. 
This  procedure  is  not  intended  to  cover 
the  submission  of  contractual  letters. 


FORMAL  SUBMISSIONS 
0 e  n  e  r  a  1  Policy 

Other  than  the  medium  of  submission, 
formal  submissions  require  the  same 
procedures  that  are  currently  in  effect. 
The  following  procedure  documents  those 
r  poets  which  are  peculiar  to  the  magnetic 
medium  process. 


Updating  Formal  Submissions 

Data  to  be  updated  shall  be  generated  as 
fell ows : 

1.  Originator.  Obtain  a  copy  of  th« 
master  diskette  from  Eata  Management. 
This  copy  shall  be  used  as  the  working 
master  for  update.  Data  Management  shall 
asign  a  control  number  revision  for 
incorporation  into  the  first  "page"  data, 
all  first  "pages"  in  last  -  fir  st  sequence 
shall  be  retained  on  each  update.  The 
updated  diskette  shall  be  separately  saved 
and  handled.  Submission  to  Data 
Management  shai1  be  handled  the  same  as 
formal  submissions  (See  above). 

2.  Data  Management.  Except  for  the 
requirement  to  maintain  separate  diskettes 
for  all  document  revisions  submitted,  the 
procedure  defined  above  is  to  be  followed: 


1.  Originator.  Prepare  documentation 

ir.  accordance  with  the  (Contract  Data 
Requirements  List  ( CD  RL )  requirements 
utilizing  blank  formatted  diskettes. 

Verified  backup  diskettes  shall  be 

prepared  by  the  originator  for  Data 
Management.  Unless  otherwise  approved  oy 
the  Engineering  .Manager,  a  complete 

printed  copy  of  the  (submittal  diskette; 
data  shall  be  prepared  fot  Data 
Management,  whether  or  not  t^e  submission 
requires  a  printed  copy.  Prior  to 
preparation  of  the  diskette,  the 
originator  shall  obtain  a  control  number 
from  data  management  and  shall  format  the 
diskette  with  that  control  number  os  its 
name.  Since  diskettes  may  contain  more 
than  one  document,  the  TD  number  shall  r.ot 
be  used  as  the  control  number.  The  first 
"page"  of  diskette  data  shall  contain: 

c  Contract  number 
o  Item  (e.g.  CD  RL  number) 
o  title 
o  revision 

o  File  name  of  the  data  on  the 
diskette 

o  Date  of  submission 
o  Originator 

o  Computer  model,  serial  number  and 
drive  number 

o  Operating  system  utilized 
o  Program  utilized  (i.e., 

Lazywri ter ,  VISICALC,  etc) 
o  Data  Management  Control  Number 
o  Diskette  Cycle  ( A , B ,  or  C) 

2,  Data  Management.  Record,  control 
and  submit  diskette  data  in  accordance 
with  current  established  prcceaures. 
Maintain  files  of  control  number,  diskette 


NOTE:  Multiple  revisions  can  be  contained 

on  the  same  diskette  t  ?  yad,  houoi»*»r 

each  submission  shall  be  on  a  new  diskette 
and  all  Drior  diskettes  shall  be  retained. 
Decisions  to  discard  prior  copies  must 
have  the  approval  of  the  Engineering  and 
Program  Manager.  Unless  specifically 
exempted,  sufficient  diskettes  will  be 
retained  to  ensure  that  copies  of  all 
revisions  are  contained  on  one  or  more  of 
the  master  diskettes. 

3.  Diskette  Cycling.  To  minimize 
the  possibility  of  loss  of  valuable  data, 
Data  Management  shall  institute  a  policy 
f  "Three  Diskette  Cycling".  The  purpose 
of  diskette  cycling  is  to  continually 
maintain  the  three  latest  revisions  of 
diskette  data  such  that  recovery  from 
either  erroneous  updates  or  diskette 
damage  is  no  more  than  one  revision  aw^y. 
A  summary  procedure  for  cycling  is  as 
follows : 

a.  Originator.  For  each  document 
(or  contract),  prepare  three 
formatted  diskettes  labeled 
(internally)  A , B  and  C.  Each 
time  an  update  is  made,  the 
originator  obtains  the  cycle  log 
from  Data  Management  and  copies 
the  updated  version  onto  the 
lowest  cycle  diskette  which  then 
becomes  th®_current  Master. 

b.  Data  Management .  Establish  and 
maintain  the  cycle  log  for  each 
diskette., 


BEST  AVAILABLE  COPY 
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File  Name?. 

File  names  shall  be  used  that  incorporate 
the  revision  as  an  extension,  e.g., 
"CDRLACC1/R01 CDRL  A001  is  at  Revision 
01  on  that  fV.  .  Each  update  shall  be 
made  by  renaming  the  current  file  and 
revision.  Where  more  than  one  revision  is 
contained  c  .  a  single  diskette,  the  new 
revision  shall  be  copied.  This  will 
facilitate  maintaining  control  utilizing 
the  directory  listing*:. 

Customer  Listings. 

hue  to  its  cost  effectiveness,  the 
customer  should  oe  encouraged  to  submit 
review  comments  by  actually  up-dating  a 
copy  of  the  document  diskette.  He  should 
identify  his  changes  by  underlining  his 
changes  or  by  writing  "*** INITIALS n  in  the 
margin.  This  process  will  expedite 
handling,  increase  accuracy  and  reduce 
cost  for  both  the  customer  and  Honeywell. 
Comments  received  this  way  must  be 
formally  acknowledged  to  the  customer  by 
sending  a  printout  of  either  the  directory 
or  the  data  of  the  updated  diskette  as 
received  by  the  Contractor 

Electronic  Submission 

There  will  be  times  when  it  will  be 
expedient  to  electronically  submit  data  ir. 
lieu  of  the  mails.  This  process  shall  be 
utiliz  d  only  for  the  submission  of 
working  ?r  advance  copies  and  she  ]  not  be 
used  a.  a  substitute  for  formal 
submission.  Advance  copy  transmission 
shall  not  occur  until  document  approval 
has  been  received.  During  transmission, 
tne  first  page  of  data  must  define  the 
document  as  an  advance  or  working 
(unofficial)  cooy  unless  specific  prior 
approval  for  alternate  handling  is 
obtained . 

INFORMAL  (ENGINEERING)  SUBMISSIONS 
General  Policy 

Informal  data  submissions  shall  be  handled 
in  accordance  with  the  current  Customer 
Engineering  Letter  (CEL)  policy  except  so 
far  a r  format  is  concerned.  The  gen.ral 
requirements  for  handling  diskette  data 
.’•hall  be  as  described  in  above.  Systems 
Engineering  is  responsible  for  the 
submission  of  such  data  and  for  performing 
‘he  diskette  handling  management  functions 
uefined  above.  Where  data  is  beirg 
de vel epee  on-line,  it  will  be  documented 
as  a  tele  cor. . 
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BASELINED  SOFTWARE  CONFIGURATION  MANAGEMENT:  AN  AUTOMATED  APPROACH 
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\  ABSTRACT 

Software  intensive  training  equipment  has  become  the  normalcy  tor  current  and  foreseeable 
uture  training  equipment  procurements.  Software  support  for  these  software  intensive  devices 
is  an  ever  'Increasing  burden  which  must  be  undertaken  by  the  most  efficient  manners  possible. 

Aid  for  modiKpation  software  support  has  been  addressed  by  software  tools  developments  that 
facilitate  modifications,  testing  and  baseline  configuration  management  of  these  software 
intense  devices.  ~*Th is  paper  addresses  the  phase  of  software  support,  configuration  management, 
which  is  often  neglected  or  given  too  litile  priority  until  the  day  of  reckoning  arrives;  the 
software  configuration  loss  or  error.  A  tightly  controlled  software  configuration  of  a  soft¬ 
ware  intensive  device  is  perhaps  as  important  as  the  configured  software  itself.  Software 
'  configuration  control  is  addressed  in  this  paper  as  a  crucial  element  of  the  life-cyc1e  support 
effort.  A  configuration  control  methodology  which  maintains  visibility  and  complete  software 
modification  audit  capabilities  is  discussed.  An  automation  of  this  methodology  is  presented 
which  affords  complete  and  accurate  control  for  modification  processing  of  a  baselined  soft¬ 
ware  configuration.  The  entire  automated  process  extends  from  the  receipt  of  a  modified 
sof uware^cofliponent/to  the  installation  and  tracking  of  a  newly  configured  and  tested  software 
release  .^Additional  ly ,  the  capabilities  for  multiple  device  software  configuration  tracking 
with  a  centralized  procedure  and  support  environment  is  explained,based  on  actual  procedures 
currently  in  use  at  the  NTEC  Software  Support  Facility  (SSF)  whiei  is  located  at  the  NTEC 
Annex  at  Herndon,  Oilando,  Florida. 

INTRODUCTION  baseline  modification  processing.  A  strong  soft¬ 

ware  configuration  management  philosophy  and  strict 
The  Software  Support  Facility  (SSF)  at  the  adherence  to  that  philosophy  is  required  if  the 

Naval  T  lining  Equipment  Center  (NTEC),  Orlando,  ever-increasing  burden  of  true  software  baseline 

Florida,  marked  a  first  at  NTEC  for  providing  soft-  support  is  to  be  effectively  addressed, 

ware  support  capability  with  in-house  resources  for 

major  training  devices. (!)  The  design  concept  of  Centralization  of  software  support  facijji y 

the  SSF  was  to  centralize  the  software  support  type  services  is  a  viable  and  efficient  coffcept  for 

functions  of  remotely  located  software  driven  the  baseline  maintenance  of  software  intensive 

training  equipment.  Utilizing  remote  communica-  training  equipment.  Further  centralization  of  con- 

tions  equipment  such  as  remote  job  entry  (RJE)  and  figuration  control/management  and  automated  program 

time  share  terminals,  on-site  support  personnel  as  processing  to  a  single  point  within  tne  SSF  envi- 

well  as  SSF  personnel  could  work  in  a  combined  ronment  ensures  commonality  of  management,  support 

effort  to  accomplish  software  modification  imple-  procedures,  and  visibility  among  the  many  and 
mentations.  Centralized  software  support  at  the  various  device  software  baselines  which  will  be 

SSF,  however,  has  evolved  from  providing  software  supported  through  the  NTEC  SSF  services.  Carried 
modification  support  to  providing  a  concentrated  to  this  level  of  centralization,  SSF  resources  and 

software  support  resource  with  which  any  user  or  manpower  need  not  be  expanded  in  a  wholesale 

activity,  regardless  of  location,  can  perform  modi-  fashion  with  each  new  software  baseline  brought 
fi-'.atlon  sjpport  e f  "orts  at  the  software  baseline  under  configuration  management, 
change  level. 

SOFTWARE  CONFIGURATION  MANAGEMENT  OBJECTIVES 
This  service  function  which  the  has  under-  AT  THE  SSF 

taken  involves  primarily  software  modification  tool 

support,  software  baseline  configuration  management  The  centralized  software  configuration  manage- 

and  program  processing,  and  a  nucleus  of  software  ment  procedures  exist  to  attain  a  definitive  set  of 
"smart"  '-ngineering  st.u**  to  assi't  any  support  objectives.  These  objectives  apply  to  the  manage- 

user  with  generic  trainer  related  software  problems  ment  of  all  configured  baselines  which  are  under 

such  as  math  modeling  and  real-time  operation.  the  direct  control  of  the  SSF  regardless  of  train- 

Modification  tool  support  involves  the  development  ing  system  or  device  type  and  embedded  computers, 

and  support  of  software  procedures  to  aid  in  the  Basically,  software  baseline  protection,  mai stain- 

change  implementation  processes  such  as  remote  com-  ability,  and  development/modification  visibility 

munications  procedures,  test  compilation  procedures  are  addressed  by  the  five  established  goals, 

for  the  various  supported  resources  within  the  SSF, 
baseline  management  support,  and  modification  in-  Objective  1 . 
formational  support.  Software  baseline  configura¬ 
tion  management  is  provided  by  the  SSF  configura-  The  SSF  configuration  management  will  estab- 

tiori  management  group.  This  group  invokes  a  lish  and  subsequently  orotect  the  integrity  of  each 

software  configuration  management  discipline  while  software  baseline  identified  for  management  under 

utilizing  an  automated  configuration  control  proce-  the  SSF's  configuration  control  procedures.  Estab- 

dure  that  allows  complete  and  accurate  control  for  lining  a  software  baseline  for  a  given  system  or 
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device  ranges  in  complexity  from  a  simple  audit  for 
software  systems  developed  under  the  guidelines  of 
MIL-STD  1644A  to  a  massive  manpower  consuming  task 
of  baseline  recreation  and  verification  as  shown  by 
the  procedure  in  figure  1;  the  former  case  being 
that  of  most  new  procurements  and  the  latter  being 
that  of  older  software  systems,  and  even  possibly 
newer  systems  where  MIL-STD  1644A  was  not  specified 
for  development  guidelines.  Initial  software  base¬ 
line  establishment  is,  therefore,  more  timely  for 
MIL-STD  1644A  developed  software,  but  not  im¬ 
possible  for  the  remaining  software  systems.  Base¬ 
line  establishment  encompasses  not  only  software 
verification  but  also  structure  preparation,  format 
arc  lysis ,  and  all  necess^^y  management  database 
development  and  loading;  in  short,  establishment  of 
all  items  identified  by  MIL-STD  1644  appendix  A. 
Protection  of  the  established  software  baseline  is 


given  the  highest  priority  of  all  management  func¬ 
tions  and  objectives  because  of  the  negative  rami¬ 
fications  that  are  possible  and  probable  when  an 
established  software  baseline  is  not  tightly  con¬ 
trolled.  Ramifications  such  as  unautnorized 
modifications,  authorized  modifications  to  an  in¬ 
correct  baseline  component,  and  even  the  loss  of 
baseline  components  are  all  too  possible.  A 
tightly  held  configuration  baseline  affords  maximum 
protection  and  confidence  of  possessing  the  correct 
baseline  for  a  given  system  or  device.  Maximum 
baseline  protection  is  accomplished  by  granting 
read-only  access  to  all  users  while  update  access 
privileges  are  held  only  by  the  software  configura¬ 
tion  management  group.  Therefore,  a  software  base¬ 
line  impact  can  only  be  accomplished  witn  configur¬ 
ation  management  knowledge  and  approval. 
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Objective  2. 

Configuration  management  will  provide  for  the 
ease  and  safety  of  implementing  modifications  to 
established  software  baselines.  Ease  of  modifica¬ 
tion  implementation  is  provided  by  the  availability 
of  software  tools,  utilities,  and  procedures  used 
to  edit,  compile,  process,  etc.,  the  baseline  com¬ 
ponent  whether  it  be  a  source/data  file  or  documen¬ 
tation  associated  with  the  change  implementation. 
Ease  of  modification  is  also  enhanced  by  the 
various  services  provided  by  the  configuration 
management  group.  Configuration  identification  is 
one  such  service.  The  currently  configured  base¬ 
line  component  is  identified  to  ensure  the  modifi¬ 
cation  is  implemented  on  the  correct  component. 

Once  identified,  availability  of  that  component  is 
sampled.  Possibly,  the  newly  identified  component 
is  already  undergoing  change.  If  not,  this  compon¬ 
ent  is  "checked-out"  or  locked  to  the  cognizant 
individual  for  worK  much  as  a  library  book  is 
"checked-out".  Inis  service  prevents  simultaneous 
modification  efforts  on  the  same  component  where 
coordination  would  not  be  <1  factor.  If  additional 
work  is  targeted  for  a  locked  component,  coordina¬ 
tion  among  all  cognizant  individuals  or  activities 
is  necessary  to  implement  any  concurrent  additional 
changes.  One  change  will  not  inadvertently  super¬ 
cede  the  other.  A  safety  factor  for  modifying  the 
configuration  baseline  is  maintained  by  allowing 
new  or  modified  components  into  the  baseline 
account  only  through  configuration  management 
services  under  strict  control  by  the  configuration 
management  group.  Once  a  new  or  modified  component 
is  accepted  into  baseline  control  its  existence  ard 
configuration  impact  are  forever  accounted  for. 
Normal  configuration  acceptance  of  a  component  re¬ 
quires  a  documented  work  effort  as  well  as  an 
acceptable  software  design  review  finding.  Un¬ 
solicited  software  component  submissions  cannot 
arbitrarily  infiltrate  the  protected  and  configured 
software  baseline  account. 

Objective  3. 

Configuration  management  will  establish  and 
maintain  configuration  modification  history/audit 
trails.  How  has  the  software  baseline  evolved  from 
the  original  baseline  establishment;  or,  how  did  it 
get  here  from  where?  The  answers  to  this  question 
are  extremely  helpful  for  maintaining  full  visibil¬ 
ity  for  the  software  maintenance  life-cycle.  Also, 
software  system  maintenance  is  aided  by  enhanced 
modification  history  analysis  capabilities.  A  con¬ 
figuration  impact  or  modification  to  the  current 
baseline  is  implemented  via  t  parent/sibling 
relationship.  An  existing  currently  configured 
element,  the  parent,  is  referenced  and  modified  to 
create  a  new  element,  the  sibling,  which  will  enter 
configuration  tracking  and  pass  through  all  appli¬ 
cable  stages  of  status  accounting  until  it  possibly 
attains  the  parenthood  status,  in  which  case  it 
would  replace  the  parent  from  which  it  was  produced 
or  seeded.  In  any  event,  a  fully  automated  his¬ 
tory/  audit  trail  is  maintained  to  track  the  evolu¬ 
tion  jf  a  configured  component  from  its  earliest 
beginnings.  This  automated  history/audit  trail 
management  allows  the  configuration  management 
group  to  selectively  report  ard/or  completely  re¬ 
construct  the  executing  configuration  of  a  software 
system  under  automated  configuration  control  at  any 
previous  pcint  of  its  baseline  history.  The  audit 
data  retrieved  in  this  manner  would  also  show  what, 


if  any,  software  components  were  in  the  test  phase 
at  that  time.  The  results  are  a  full  life-cycle 
visibility  for  ail  baselined  software  under  auto¬ 
mated  configuration  control. 

Objective  4. 

Configuration  management  will  provide  access 
services  for  baseline  source/data  files,  listings, 
and  all  applicable  support  data.  All  currently 
active  device  software  configuration  baselines  are 
maintained  in  a  mounted,  on-line  status  ensuring 
ready  baseline  access  for  all  cognizant  modifica¬ 
tion/development  activities  and  individuals.  The 
entire  software  source/data  baseline  for  all 
supported  software  systems  is  available  for  read¬ 
only  access  to  the  cognizant  individual  maximizing 
access  ease  while  minimizing  potentials  for  base¬ 
line  violations.  For  modification  implementations 
a  copy  of  the  configured  baseline  component  is 
easily  retrieved  into  the  implementor's  work 
account  where  changes  can  be  safely  made.  For 
software  component  development  the  implementor 
simply  develops  the  component  in  the  work  account. 
At  the  comoletion  of  modification  implementation  or 
new  component  development  the  new  software  compon¬ 
ent  is  accepted  into  the  configured  baseline 
account  only  by  the  configuration  management  group 
after  formal  acceptance  criteria  have  been  met, 
i.e.,  software  design  review  and  quality  assurance 
approval . 

All  listings  such  as  compi lation/assembly 
listings  that  are  produced  during  automated  con¬ 
figuration  program  processing  will  be  cataloged  and 
archived  for  rapid  recovery  upon  request.  Life- 
cycle  support  for  a  software  baseline  includes  the 
efficient  tracking  and  storage  or  all  supportive 
data  produced  during  the  management  of  that  base¬ 
line.  Therefore,  all  such  listings  and  data  files 
are  retrievable  from  the  configuration  management 
qroup  upon  request  in  a  timely  manner. 

Objective  5. 

Configuration  management  will  provide  maximum 
visibility  for  all  software  modification/develop¬ 
ment  efforts  managed  under  the  automated  configura¬ 
tion  control  procedures.  Automate!  configuration 
management  tracks  a  software  system  baseline  from 
initial  baseline  establishment  to  the  currently 
configured  state.  Automated  configuration  manage¬ 
ment  cracks  and  manages  all  modif ication/develop- 
ment  activities  applied  against  a  supported 
configured  baseline.  Automated  configuration 
management  relates  the  entire  software  baseline 
life-cycle  evolution  via  parent/sibling  relation 
chains.  These  facts  enable  the  automated  configu¬ 
ration  structure  of  a  supported  software  system 
baseline  to  yield  various,  informative  reports  con¬ 
cerning  any  phase  of  any  or  all  components  of  any 
era  of  the  life-cycle  of  the  software  baseline;  in 
short,  complete  management  visibility  of  the  ever 
changing  configured  software  baseline.  This 
visibility  is  available  as  soon  as  the  software 
baselina  transitions  to  automated  configuration 
management  and  remains  available  for  the  life  or 
the  supported  baseline.  For  maximum  control  and 
visibility  software  baseline  development  as  well  as 
baseline  nodi f ication  should  be  managed  via  the 
automated  configuration  control  procedures. 
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AUTOMATED  SOFTWARE  CONFIGURATION  ENVIRONMENT 


As  software  intensive  devices  continue  to  in¬ 
crease  the  training  equipment  inventory  and  as 
their  operational  locations  continue  to  be 
scattered  according  to  the  operational  needs  of 
their  users,  the  rationale  for  a  single  centralized 
software  support  facility  becomes  increasingly 
stronger.  Centralization  of  software  support 
does  not  infer  locale  central  to  all  devices  nor 
does  it  infer  a  large  central  cadre  of  application 
programmers  and  analysts  to  supply  the  software 
support  activity  functions  for  all  the  various 
software  intensive  training  equipment.  The  mission 
of  the  centralized  software  support  facility  is  to 
supply  a  central  set  of  resources  to  support  the 
software  modification/development  activities  wher¬ 
ever  and  with  whomever  they  may  be  located.  These 
software  support  resources  include,  but  are  not 
limited  to,  all  necessary  hardware,  software,  base¬ 
line  storage  and  management,  procedures,  and 
expertise  needed  to  comprise  an  efficient  and 
responsive  software  support  facility  environment 
whose  prime  services  are  centralized  program  pro¬ 
cessing  and  configuration  management. 

Host  Configuration  Processor 

The  strengthening  rationale  of  centralized 
software  support  also  applies  to  a  centralized 
software  configuration  management,  centralized  to 
the  extent  of  common  procedures,  commcn  database 
structure,  consolidated  baseline  on-line  storage, 
and  a  common  point  of  contact  for  all  modification/ 
development/configuration  management  users:  the 
host  conf "’guration  processor.  The  host  processor 
performs  aM  user  communications,  both  remote  and 
local  time  share  and  remote  Job  entry  (RJE),  auto¬ 
mated  configuration  database  management,  baseline 
storage  and  control,  and  automated  distributed 
processing  management  for  all  supported  software 
system  baselines.  In  the  automated  configuration 
environment  the  host  processor  is  normally  the  only 
external  communications  link  with  the  software 
support  facility  services.  All  users  interface  a 
common  editor  for  source  baseline  modification/ 
development,  a  common  set  of  support  services  and 
tools,  a  common  set  of  information  resources,  and 
most  importantly  a  common  procedural  base  fo*  all 
supported  and  configured  software  system  baselines. 
Program  processing  and  management  and  device  depen¬ 
dent  functions  are  normally  transparent  to  the 
software  support  facility  user. 

pist r i bu t ed.  Processin (j_  Network 

Figure  2  illustrates  the  distributed  network 
through  which  the  generic  configuration  processing 
system  is  able  to  complete  the  program  processing 
functions.  The  host  configuration  processor  is 
merely  an  automated  manager  ar.j  does  not  perform 
any  native  mode  device  dependent  program  processing 
such  as  compilations,  assemblies,  or  link  edits. 
Since  these  functions  are  distributed  among  various 
satellite  device  dependent  systems  the  user  inter¬ 
face  remains  responsive  and  user  friendly  while  me 
common i cat  ions  unencumbered  satellite  systems  of 
the  network  are  busily  and  efficiently  performing 
their  native  mode  program  processing  functions. 
Several  program  processing  functions  may  be  active 
concurrently  across  the  bank  of  satellite  proces¬ 
sors,  none  interferring  with  the  others. 
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Figure  2.  Distributed  Configuration  Manage¬ 
ment  and  Program  Processing  Network 


The  host  configuration  processor  manages  the 
distributed  processing  functions  by  dynamically 
creating  procedure  or  job  streams  that  will  execute 
on  the  target  satellite  processor  to  accomplish  a 
desired  program  processing  function.  These  dynamic 
job  streams  are  composed  from  job  stream  seed 
tables  which  define  the  functions  and  procedures  to 
be  executed.  Configuration  items  by  name  and  com¬ 
ponent  which  are  managed  within  the  automated  con¬ 
figuration  control  database  are  sv/apped  for 
identifiable  labels  found  within  the  job  stream 
seed  table  entries.  Once  the  procedure  or  job 
stream  composition  is  complete,  communication  pro¬ 
cedures  utilizing  an  emulation  of  HASP  transfer  the 
job  stream  file  to  the  target  processor  for  execu¬ 
tion.  With  these  automated  procedures  any  target 
or  device  dependent  computer  can  be  supported  with¬ 
in  the  configuration/program  processing  structure 
by  simply  constructing  the  proper  job  seed  tables 
to  accomplish  the  functions  of  program  processing 
necessary  for  that  computer. 

Since  the  program  processing  satellite  compu¬ 
ter  systems  will  not  be  performing  software  base¬ 
line  management  and  storage,  only  a  minimal 
operational  architecture  is  required.  Functional 
requirements  include  resources  capable  of  executing 
a.  normal  batch  or  procedure  stream  environment  with 
remote  HASP  communi cations  ability  for  bidirection¬ 
al  host  configuration  processor  file  transfers. 
Additional  system  software  support  capabilities  may 
be  needed  for  operating  system  and  software  tool 
development  support  at  the  satellite  processor 
level . 

Configurable  So f t wa re_ Str uc t u_» e 

All  device  software  systems  that  are  to  be 
configured  and  managed  by  the  automated  configura¬ 
tion  control  procedures  must  be  structured  in  a 
uniform  manner  so  as  to  allow  a  gener:  set  of 
automated  software  tools  and  procedures  to  operate 
by  a  common  set  of  rules.  M I L -STD  1644A  is  a 
military  standard  approved  for  use  by  the  Naval 
lraining  Equipment  Center  and  U  available  for  use 
by  all  Departments  and  Agencies  of  the  Department 
of  Defense  for  trainer  system  software  engineering 
requirements. (2/  This  standard,  with  its  appendix 
A  specifically,  establishes  the  common  set  of  rules 
which  will  enable  the  automated  configuration  cjn- 
trol  procedures  to  manage  the  many  ana  various 
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device  software  systems  developed  to  this  standard 
regardless  of  their  origin,  function,  or  embedded 
equipment.  Appendix  A  defines  the  construction  of 
the  device  system  software  so  that  incremental 
modifications  may  be  readily  performed  and 
managed.^)  Additionally,  the  requirements  for 
various  data  item  deliveries  are  established  which 
essentially  make  up  the  initial  automated  configu¬ 
ration  control  database  load. 

Figure  3  depicts  the  segmented  task  construc¬ 
tion  required  by  appendix  A.  Each  program  struc¬ 
ture  is  divided  or  segmented  into  defined  computer 
n>diiory  areas  called  windows.  The  beginning  address 
of  each  window  relative  to  the  beginning  of  the 
entire  program  area  is  tracked  within  the  configu¬ 
ration  management  database.  The  size  of  each 
window  is  selected  so  as  to  contain  a  logical 
module  or  a  group  of  executable  modules  together 
with  an  initial  spare  memory  area  of  fifty  percent 
of  the  allocated  window  area. (2)  This  segmented 
structure  allows  each  window  area  of  the  entire 
program  to  be  independently  modifiable  via  overlay¬ 
ing.  This  concept  greatly  enhances  the  incremental 
modification  efforts  by  permitting  the  processing, 
i  e.,  link  editing,  transmitting  (RJE),  etc.,  of 
only  that  part  of  the  program  which  is  effected  by 
the  increment  change.  Also,  the  modification  risk 
factor  is  lessened  by  the  need  for  only  handling 
that  section  of  the  program  which  is  to  be  modified 
vice  the  usual  need  for  handling  the  entire  program 
for  each  change  implementation.  Once  the  configu¬ 
ration  control  database  is  loaded  w.th  the  neces¬ 
sary  data  relative  to  a  configured  software 
baseline,  construction  of  the  entire  program  or  any 
of  its  defined  window  areas  as  overlays  can  be 
accomplished  by  the  automated  configuration  program 
nrocessing  system. 
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Gpe r a  t  i  o na_l  P rocedu res 

An  effective  modification  and  configuration 
management  function  requires  a  defined  operational 
procedure  and  strict  adherence  to  that  procedure. 
This  pap0"  addresses  an  automated  configuration 
management  environment  whose  implementation  none 
the  less  requires  such  an  operational  procedure. 
The  SSF  procedures  described  herein  fo*  automated 
configuration  management  are  implemented  in  a 
circular  fashion  such  that  the  modification/con- 


fiouration  management  procedural  flow  terminates  at 
the  step  where  initial  authorization  for  change 
implementation  originates. 

A  software  baseline  modification  process 
begins  when  device  modification  management  autho¬ 
rizes  the  implementation  of  a  device  modification 
which  impacts  the  software  baseline.  The  cognizant 
software  support  activity  performs  a  preliminary 
analysis  to  determine  the  software  components 
effected  by  name.  The  SSF  configuration  management 
group  is  requested  to  begin  configuration  control 
of  the  modification  implementation.  Configuration 
management  identifies  the  currently  configured  re- 
vision  of  the  modules  identified  and  reports  as  to 
their  availability  for  modification.  If  a  change 
is  currently  being  implemented  to  any  of  the 
identified  modules  then  the  new  modification  to 
that  module(s)  must  either  be  deferred  or  forwarded 
to  the  individual  or  group  who  has  current  cogni¬ 
zance  for  modification.  If  the  modules  are  avail¬ 
able  then  configuration  management  locks  the 
modules  to  the  requesting  cognizant  individual  or 
group  so  as  to  prevent  uncoordinated  parallel  modi¬ 
fication  efforts.  A  new  revision  level  for  the 
modified  module  is  also  assigned  at  this  time.  The 
cognizant  software  support  activity  performs  modi¬ 
fication  design,  source  module  change  implementa¬ 
tion,  and  the  preliminary  documentation  updates. 
Next,  a  software  design  review  is  held  to  ensure 
good  software  design  practices  are  used,  to  review 
documentation  updates  and  test  procedures,  to 
ensure  assigned  configuration  data  items  are 
correct  and  to  ensure  that  the  software  charge 
fully  addresses  the  original  change  request.  If 
the  results  of  the  software  design  review  prove  to 
be  unsatisfactory  then  the  software  support 
activity  continues  with  the  implementation  process 
until  a  successful  ensign  review  result  is 
attained.  After  a  successful  design  review,  the 
SSF  configuration  management  group  accepts  the  new 
module(s)  produced  from  the  previous  baselined  ver¬ 
sion  into  the  configured  software  baseline  account. 
Once  configuration  control  has  possession  of  the 
new  software  component,  the  automated  configuration 
Management  system  is  utilized  to  produce  a  device 
loadable  software  component,  i.e.,  an  overlay, 
ready  for  device  level  testing.  Arrangements  .ire 
made  by  the  software  support  activity  with  the 
custodian  o*'  the  particular  training  device  which 
is  used  as  the  test  bed  for  modification  software 
tests.  Before  the  newly  created  modification  soft¬ 
ware  is  released  from  SSF  configuration  conlrc'  a 
pre-checkout  status  update  is  made  to  the  automated 
configuration  control  database.  This  status  entry, 
described  in  more  detail  later,  indicates  that  the 
configured  nodi f ication  software  has  been  released 
for  device  level  testing  but  is  not  part,  of  the 
current  executable  oaseline.  The  modification 
software  which  is  in  overlay  module  form  is  re.idy 
for  transfei  to  the  device  test  site  either  via 
remote  job  entry  ( R JL )  terminal  reception  or  magne¬ 
tic  tape  and  the  U.S.  Postal  Service.  Additicna1- 
ly,  detailed  test  procedures  are  provided  with  the 
software  release  under  test.  At  this  point  either 
a  field  engineering  representative  (FIR)  of  NIK  or 
an  on-site  menber(s)  of  the  cogn.zant  software 
support  activity  tests  the* nodi f lea t ion  software  by 
overlaying  the  current  executable  software  on  the 
training  device  in  a  test  environment .  If  the 
operational  test  fails  the  configuration  management 
group  is  notified  and  the  configuration  control 
database  is  updated  to  show  that  the  test  modified- 
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tion  software  is  rejected  in  which  case  the  modifi¬ 
cation  work  effort  would  be  returned  to  the  cogni¬ 
zant  individual  or  activity.  When  a  successful 
test  is  performed  the  modification  software  is 
qualified  for  release  as  an  official  Training 
Equipment  Change  Directive  (TECD).  The  TECD  will 
be  supplied  by  device  modification  management  to 
the  applicable  device  sites  as  a  kit  containing  the 
pre-tested  modification  software,  installation  and 
test  procedures,  and  all  applicable  documentation 
updates.  Once  the  device  site  support  personnel 
have  installed  and  tested  the  newly  released  soft¬ 
ware,  NTEC  is  notified  via  a  Record  of  Completion 
form  suPDlied  with  the  kit.  The  SSF  configuration 
management  group  is  notified  and  again  the  config¬ 
uration  control  database  is  updated  to  show  that 
now  the  modification  software  is  approved  for 
training  and  is  currently  configured  within  the 
executable  baseline.  All  source  baseline  compon¬ 
ents  which  were  previously  locked  for  change  imple¬ 
mentation  are  automatically  unlocked  at  the 
approval  status  update.  Modification  management  is 
notified  of  the  baseline  update  and  the  modifica¬ 
tion/configuration  control  operational  cycle  for 
that  software  change  is  complete. 

Several  software  modifications  may  be  in 
various  stages  of  this  cycle  simultaneously.  Also, 
more  than  one  software  support  aw-ivity  may  be  per¬ 
forming  change  implementations  to  the  same  or 
various  other  device  software  baselines.  In  any 
event,  the  same  procedures  and  the  same  management 
is  being  performed  for  all  active  modification 
efforts . 

Software  Baseline  Vi  Sibil ity 

Visibility  of  the  ever  changing  state  of  a 
configured  software  baseline  significantly  aids  the 
baseline  software  management  t'.sk.  If  the  status 
of  a  baseline  is  obscure  or  difficult  to  assess  no 
one  benefits;  not  the  software  support  activities 
and  especially  not  device  modification  management. 
The  automated  configuration  management  procedures 
are  designed  to  allow  the  display  of  the  state  of 
any  configured  baseline  on  demand.  The  configura¬ 
tion  history/audit  trail  capabilities  are  imple¬ 
mented  via  a  parent/sibling  approach  with  eight 
levels  of  status  for  the  various  life-cycle  stages 


a  configuration  component  may  traverse.  Each 
occurrence  of  a  new  status  condition  is  recorded 
within  the  configuration  management  database  along 
with  the  date  of  the  occurrence.  All  previous 
status/date  entries  remain  available  within  the 
database  and  are  recorded  in  chronological  order  of 
occurrence.  The  parent/sibling  relationship  of  the 
baseline  components  is  maintained  y  a  chain  key 
linkage.  Each  component  database  entry  as  it  is 
created  by  a  baseline  modification  effort  is 
stamped  with  the  database  key  of  the  parent  compon¬ 
ent  from  which  it  v/as  produced.  In  turn,  the 
parent  component  database  entry  heads  a  chain  key 
linkage  which  points  to  its  siblings  in  the 
sequence  they  were  created.  As  each  new  sibling  is 
identified  its  key  is  placed  at  the  bottom  of  its 
parent's  sibling  chain. 

The  combination  of  interrelationships  among 
the  parent/sibling  chain  ana  the  various  life-cycle 
status  entries  for  each  modularized  configuration 
component  yield-*  i  capability  for  on-demand  config¬ 
uration  status  displays  and  reports.  Figure  4  is 
an  example  of  a  software  modification  status  dis¬ 
play  produced  by  the  automated  configuration 
management  system.  The  left-hand  field  of  this 
display  identifies  by  name  and  database  key  the 
structured  window  sequence  of  a  single  segmented 
task  or  program  load.  Ihe  configuration  database 
entries  whose  keys  are  referenced  here  contain  the 
actual  modular  construction  of  each  window.  The 
next  field  labeled  "Approved"  represents  modifica¬ 
tion  software  or  overlays  that  have  been  developed, 
applied,  and  approved  for  execution  in  the  training 
environment  and  arc  part  of  current  baseline  for 
this  task.  These  "approved"  overlays  are  executing 
in  place  of  their  "installed"  counterparcs.  The 
"In  Check-Out"  field  identifies  modification  soft¬ 
ware  that  has  been  released  by  the  configuration 
management  group  for  device  level  testing.  This 
software  now  physically  resides  on  the  test  device 
for  testing  only.  The  last  field  identifies  modi¬ 
fication  software  that  is  ready  for  testing  against 
this  task  but  has  as  yet  to  be  transferred  to  the 
device  test  site.  Additional  display  capabilities 
show  the  current  and  previous  baseline  modification 
activities  to  any  modular  level.  Any  previous 
point  in  the  baseline  life-cycle  history  can  be  in¬ 
quired  upon  for  configuration  content  and  modifica- 
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*Conf iguration  Matrix,  14  Task  Conf. 

1  -  Conf.  List  Window  Key  &  Sequence, 

Task  1. 

2  -  Configuration  List,  Task  H. 

;  3  -  Conf.  History  Data  Entry  (Overlay 
List),  Keyed  Entry  1000. 

Source  Data/Address 
4  -  Keyed  Entry  1046. 
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-  Each  Matrix  reflects  the  configuration  of  a 
task  set  release  for  a  configuration  year. 

Task  References 

Task  Map  Names 

T^sk  File  Names 

Task  Configuration  List 

-  Each  Task  Configuration  List  contains  the 
window/overlay  keys  and  their  sequence  which 
comprise  the  associated  task  image. 

OCN  (Overlay  Control  Number) 


Figure  5.  Tri-Level  Configuration  Database 
Structure 


tion  status.  At  any  point  in  time  the  current 
executing  baseline  is  merely  a  request  away  with 
the  SSF  automated  configuration  management  system. 

Autome ted  Software  Configuration  Management  Tool 

The  software  configuration  management  system 
revolves  about  a  single  configuration  control  tool. 
This  software  tool  performs  the  configuration  and 
program  processing  management  functions  for  any 
number  of  training  device  baselines.  Configuration 
management  provided  by  this  tool  (or  the  various 
supported  software  baselines  is  implemented  with  a 
three  level  database  structure  shown  by  figure  5. 
The  outer  level  consists  of  a  matrix  file  which  re¬ 
lates  a  physical  device  by  serial  number  to  a  base¬ 
line  configuration  level.  A  scm'S  of  matrices 
each  representing  a  configuration  year  will  prolif¬ 
erate  throughout  the  life-cycle  of  the  device. 

Each  record  entry  within  the  matrix  file  defines  a 
single  task  or  load  program.  Multi -program  soft¬ 
ware  baselines  have  as  many  matrix  record  entries 
as  the  number  of  separate  tasks.  Each  entry 
references  the  task  name,  task  load  map,  and  task 
configuration  list  database  key.  This  database  key 
is  the  linkage  from  the  outer  data  structure  to  the 
second  level.  The  configuration  list  contains  a 
sequence  oi  third  level  data  structure  keys  as 
shown  by  figure  6.  Each  of  these  third  level  keys 
point,  to  a  cent  iguration  history  dataset  entry 
(figure  7)  which  reference  the  actual  modular  con¬ 
structs  for  every  Usk  window  segment.  The  config¬ 
uration  list,  therefore,  relates  the  specific 
windows  and  window  sequence  which  comprise  the  task 
associated  witn  the* list  found  in  the  matrix  entry 
for  that  task  reference.  A  status  entry  dataset 
which  is  at  the  same  structural  level  as  the 
history  dataset  contains  the  accumulated,  chrono¬ 
logically  ordered  status  entries  associated  with 
the  life-cycle  of  each  window/overlay  component  re¬ 
presented  within  the  configuration  history  dataset. 
Figure  8  shows  the  eight  possible  status  er tries 
which  a  component  may  acquire. 
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Figure  6.  Configuration  List  Dataset 


An  active  software  baseline  is  continuously 
evolving.  The  automated  configuration  processing 
tool  described  here  maintains  the  configuration 
version  base  on  a  yearly  cycle.  Each  yearly  cycle, 
the  configuration  year,  is  referenced  by  a  separate 
matrix.  The  configuration  cycle  begins  with  a  con¬ 
figuration  release  which  provides  a  new  set  of  load 
programs  and  all  base  software  (i.e.,  data,  proce¬ 
dures,  utilities,  etc.)  to  each  supported  device  as 
the  installed  software  configuration  base.  During 
this  year  software  modifications  are  implemented 
and  approved  as  overlays  to  the  installed  base. 
Therefore,  current  executable  configuration  at  any 
given  point  in  time  is  the  installed  base  plus  any 
approved  overlays.  At  approximately  one  month 
prior  to  completion  of  the  current  configuration 
year,  the  tool  invokes  a  pre-freeze  condition  where 
new  modification  software  releases  are  prohibited 
to  allow  a  full  testing  period  for  current  modifi¬ 
cation  software  approved  for  training  on  the  de¬ 
vice.  Next,  the  tool  performs  configuration  freeze 
which  constructs  new  configuration  lists  (figure  9) 
from  the  combination  of  currently  installed  and 
approved  wiridow/overlays.  New  matrices  are  pro¬ 
duced  which  reference  the  newly  created  lists  and 
the  new  task  names  that  will  be  produced  from  these 
new  lists.  When  the  freeze  process  is  completed  a 
new  installed  software  configurer  Ion  base  is  ready 
for  release.  The  next  configuration  year  is  be¬ 
ginning  and  the  modi fi cat  ion  process  begins  anew. 
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Software  Development  Environment 


Window/Overlay  Life  Cycle  Status 
Status  Entry:  Matrix  ID  +  Status  +  Date 
10  PN  YRMODA 

10  UT 

10  CK 

20  CK 

10  AP 

20  AP 

Typical  Status  Sequence  for  a  Single  OCN: 

PN— >UT— »CK-»AP->IN  (Matrix  f  1  )-*RF  (Matrix  +  2) 
^RJ  ^RJ  ^RJ  ^RM 


PN  -  Creation  ^ending 
UT  -  Untested 
CK  -  Check-Out 
AP  -  Approved  (cemented) 
RJ  -  Rejected 


IN  -  Installed  (Task 
Image) 

RP  -  Replaced 
RM  -  Removed  (without 
replacement) 


The  SSF  service  environments  discussed  herein 
have  only  addressed  the  modification  of  existing 
software  baselines.  A  software  baseline  as  used  by 
the  automated  configuration  management  system  must 
be  established  prior  to  modification  support.  If 
the  initial  baseline  was  actually  developed  using 
this  system,  then  transition  from  development  to 
support  would  become  completely  transparent  as  the 
baseline  would  already  exist  at  completion  of 
development.  Baseline  control  and  management  would 
begin  at  the  very  onset  of  development.  Just  as 
importantly,  the  same  visibility  available  during 
the  maintenance  phase  of  the  software  life-cycle 
woul.'  also  be  available  during  the  development 
phase. 


Figure  8.  Window/Overlay  Statu?  Structure 


The  software  tool  providing  this  management 
capability  is  table  driven  and  automated  to  the 
extent  of  removing  as  many  human  interactions  as 
possible.  This  results  in  reduced  human  induced 
errors  and  reduced  operator  requirements  for 
trainer  software  configuration  management.  The 
tool  is  initiated  by  specifying  a  device  identifi¬ 
cation  file  which  contains  the  device  name  and 
serial  number,  the  database  descriptor  modjle  name 
which  identifies  the  configuration  database  for 
this  device  and  the  matrix  identifier  of  the 
current  configuration  installed  base.  Also,  a 
command  file  or  input  device  is  speciiied  from 
which  the  automated  tool  will  receive  commands  and 
sub-directives.  These  commands  direct  the  tool  to 
perform  the  various  functions  necessary  to  perform 
configuration  and  program  processing  management. 
However,  once  the  functions  are  specified  their 
execution  is  completely  automatic.  This  automated 
tool  allows  maximum  flexibility  for  the  construc¬ 
tion,  maintenance,  and  software  configuration 
trackabil ity  of  one  or  ,  ay  number  of  training 
device  baseline  versions  and/or  revision  levels. 

By  simply  maintaining  the  device  identification 
tables  any  trainer  combination  or  multiple  simul¬ 
taneous  combinations  are  possible  and  manageable. 
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Figure  9.  Task  Configuration  Freeze 


The  development  activity  would  begin  with  task 
or  program  function  definition  and  then  proceed  to 
the  modular  and  structural  definition  phases.  At 
this  point  MIL-STD  1644A  requirements  begin  to 
effect  the  establishment,  phase  of  the  developing 
baseline  and  it  is  at  this  point  when  the  automated 
configuration  management  system  could  be  imple¬ 
mented  as  an  efficient  and  effective  baseline 
management  procedure  for  the  duration  of  the  soft¬ 
ware  life-cycle.  Tool  implementation  would  require 
an  initial  configuration  matrix  file  load.  The 
configuration  history  dataset  would  be  automatical¬ 
ly  loaded  as  the  window  constructs  are  identified 
and  p/rcessed.  From  a  planned  and  engineered  soft¬ 
ware  system  base  design  a  configured  software  base¬ 
line  would  arise. 


SUMMARY 

The  NTEC  Software  Support  Facility  has  evolved 
into  a  concentrated  central  software  support  re¬ 
source  with  which  any  modification  or  development 
activity  can  effectively  implement  software  base¬ 
line  changes  while  remaining  within  the  guidelines 
of  a  strong  configuration  management.  The  auto- 
,nated  configuration  management  system  at  the  SSF 
provides  software  baseline  protection  and  control, 
ease  of  baseline  maintenance,  and  full  development/ 
modification  visibility  during  the  support  baseline 
life-cycle.  This  automated  and  centralized  envi¬ 
ronment  permits  a  common  contact  point,  a  common 
set  of  procedures,  and  a  single,  common  management 
of  all  supported  device  baselines  under  SSF  config¬ 
uration  management.  MIL-STD  1644A  is  the  primary 
requirement  tnat  provides  a  common  set  of  rules 
enabling  various  established  baselines  from  various 
origins  to  be  managed  by  a  single  generic  confiqur- 
ation  processing  system.  Older  device  baselines 
that  were  not  designed  by  the  requirements  of  MIL- 
STD  1644A  may  require  additional  effort  to  modify 
tneir  structure  and  to  provide  the  necessary 
support  data  for  the  automated  configuration 
nenagement  system. 

The  combined  irivdif ication  management  and  con¬ 
figuration  management  procedures  of  the  NTEC  SSF 
environment  provide  a  well  defined  procedural  base 
for  software  baseline  change  implementations.  The 
strong  software  configuration  philosophy  maintained 
by  the  SSF  configuration  management  group  will  help 
ensure  that  NTEC  can  meet  the  challenge  presented 
by  the  increasing  inventory  of  software  intensive 
training  equipment.  Software  li?e-cycle  manage¬ 
ability  and  visibility  will  be  greatly  enhanced 
while  the  costs  oi  these  most  desirable  capabil- 
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ities  are  reduced.  The  requirement  of  an  effec¬ 
tive,  unified  management  ?n<j  control  system  for 
the  compliant  software  baseline,  cradle  to  grave, 
has  been  identified  and  achieved. 
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USING  SOFTWARE  DEVELOPMENT  FACILITIES  TO  IMPROVE  SOFTWARE  QUALITY 


S.  J.  TRENCANSKY,  D.  W.  MEEHL  AND  H.  C.  ROMINE 
THE  SINGER  COMPANY,  LINK  FLIGHT  SIMULATION  DIVISION 
BINGHAMTON,  N.Y. 


"A  major  problem  exists  in  the  development  of  current  state-of-the-art 
weapons  system  trainers  in  the  quality  of  the  software  provided.  Since 
the  software  for  a  given  trainer  is  typically  generated  on  the  deliver¬ 
able  computer  system  for  the  trainer  a  wide  variance  in  the  tools 
available  to  support  the  design  and  development  of  the  software  exists 
from  project  to  project. 

■The  specification  requirement  of  a  high  order  language  (FORTRAN)  has 
led  to  the  evolution  of  software  development  facilities  at  various  sim¬ 
ulator  manufacturers.  While  these  have  immediate  impact  in  reducing 
the  cost  of  the  software  produced  and  aid  in  maintaining  schedule  on 
the  ongoing  programs,  their  most  significant  impact  is  that  the  soft¬ 
ware  delivered  using  them  is  significantly  improved. 

The  paper  explains  what  was  experienced  when  the  SDF  concept  was  appli¬ 
ed  to  some  current  contracts. 

/\\ 


1 .  INTRODUCTION  ^ 

The  computer  programs  (software)  developed 
for  current  state-of-the-art  weapon  system 
trainers  (simulators)  have  been  typically 
generated  on  the  deliverable  computer 
system,  and  thus  have  been  subject  to  wide 
variance  in  the  available  software  design, 
development,  test  and  configuration  con¬ 
trol  support  programs  (tools)  from  project 
to  project. 

Varying  levels  of  available  tools  have 
been  considered  normal,  especially  when 
assembler  languages  were  used  on  varying 
brands  of  computation  equipment.  The 
introduction  of  high-order  languages 
(HOL),  predominantly  FORTRAN,  as  a  speci¬ 
fication  requirement  has  led  to  the  evo¬ 
lution  of  Software  Development  Facilities 
( SDF ) .  The  SDF  concept  was  developed  with 
three  major  objectives: 

a.  Reduce  the  cost  of  the  software 
produced 

b.  Maintain  or  accelerate  schedule 
performance  of  on-going  projects 

c.  Maintain  precise  configuration 
control  and  management 

These  objectives  have  been  fully  realized, 
and  the  SDF  provides  deliverable  software 
of  significantly  improved  quality. 

The  Link*  SDF  has  been  designed  to  provide 
conf  iguratio!.  control  and  management 
(CC/M)  in  u  user  friendly  working  atmo¬ 
sphere.  Systems  Engineering  and  software 
development  personnel  use  conveniently 
located  interactive  terminals  during  all 
phases  (design,  development,  test,  main¬ 
tenance,  etc.)  of  unclassified  projects. 


*A  trademark  of  The  Singer  Company 


The  principal  features  of  this  interactive 
capability  are: 

o  Prepare  and  edit  source  code 

o  Compile  and/or  assemble  the 

edited  source  code 

o  Prepare  and  edit  Math  Model  Test 
(MMT)  inputs 

o  Create  a  Software  Change  Request 
(SCR)  for  source  code  modified 
at  the  SDF 

2.  SDF  TOOLS 

Three  basic  categories  of  tools  exist  in 
all  SDFs  (see  Figure  1): 

a.  Tools  supplied  by  computer  and 
software  vendors 

b.  Tools  developed  by  the  software 
manufacturer  specifically  for 
the  SDF,  normally  via  IR&D 
programs 


c.  Tools  developed  under  contract  as 
deliverable  contract  end  items 


Figure  1 

Software  Development  Facility  Tools 
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o  CONFIGURATION  CONTROL  AND  MANAGEMENT 
o  COST  AND  SCHEDULE  MONITORING 


<tpn 


|o  AUTOMATED  SEARCH  FOR 
(  X  1ST  INC.  DESIGNS,  SPEC'S, 
DATA,  DOCUMENTATION,  ETC. 

|n  REQUIREMENTS  TRACKING 
HJsOI  s 


o  INTERACTIVE  SOURCE  AND 
DATABASE  DEVELOPMENT 

o  EDITORS 
o  MATH  MODEL  TEST 
o  AUTO  FLOWCHART 


*  COMPUTER-COMPUTER  TRANSFER 

>  DATABASE  SAVE/RESTORE 

>  SUBROUTINE  CALL  TREE  GENERATOR 

o  MULT  I -SIMULATOR  DEVELOPMENT  IN  ONE  COMPUTER 
n  PEAI  TIME  DFBUG 
o  TEST  CASE  VERIFY 

>  MODULE  RELINK  ON  DATABASE  CHANGE 
)  RECORD/PLAYBACK  LIST  GENERATOR 

i  III  [  COMPARE 

.  disassembler 


o  SYMBOLIC  RECORD, 
LOG  AND  PI  OT 


Figure  2  SDF  Capabilities  Used  During  Simulator  Development 


2.1  VENDOR  SUPPLIED  TOOLS 

The  Link*  SDF  is  equipped  with  a  large 
number  of  proprietary  tools  supplied  by 
the  computer  vendor  such  as  operating 
systems,  data  base  management  systems, 
terminal  systems,  compilers,  assemblers, 
loaders,  librarians,  etc.  The  high-order 
languages  (HOL's)  supported  include: 
FORTRAN,  COBOL,  BASIC.,  and  PASCAL.  In 
addition  Link  has  purchased  special  pur¬ 
pose  proprietary  software  for  use  during 
specific  phases  of  a  simulator's  life 
cycle.  Examples  of  this  software  include: 
a  lenu  design  package  used  by  visual  en¬ 
gineering  during  the  proposal  and  con¬ 
ceptual  design  phases,  a  microcomputer 
cross  assembler  and  emulator,  sophisti¬ 
cated  print  and  plot  packages,  a  cost  and 
schedule  monitoring  package  providing 
critical  path  and  other  automated  manage¬ 
ment  capabilities  and  a  comprehensive 
system  providing  full-field  search  and 
selected  output  of  cited  data  within  the 
Engineering  Information  System  (EIS). 

Figure  2  shows  the  traditional  development 
cycle  of  a  simulator  through  customer 
acceptance  (verification).  At  each  stage 
of  this  process,  the  SDF  enhances  our 
ability  to  quickly  and  accurately  perform 
the  indicated  task. 


2.2  LINK  DEVELOPED  TOOLS 

Extensive  systems  and  tools  have  been 
developed  to  control  and  enhance  every 
access  to  the  SDF.  These  capabilities  may 
be  best  understood  through  an  explanation 
of  key  features: 


o  COMPILATION  COMMANDS 


Single  commands  to  accomplish 
the  compilation,  library  edit, 
and  production  of  various  op¬ 
tional  types  of  output,  as 
discussed.  Examples:  ASSEMBLE, 
FORTRAN . 

o  MULT I -SIMULATOR  SUPPORT  PACKAGE 

Software  that  interfaces  to  CC/M 
managed  software.  It  basically 
allows  copies  to  be  made  from 
CC/M  disks,  while  prohibiting 
unauthorized  changes.  The  copy 
can  be  used  to  develop  the 
required  version,  along  with  the 
applicable  database  (Symbol 
Dictionary)  entries  applicable 
to  the  specific  project. 


/ 
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o  OPERATOR  COMMAND  STURCTURE/ 

VENDOR  UTILITY  COMMANDS 

Tools  developed  to  utilize 
vendor  software,  formalizing  the 
way  is  to  be  used  by  each  opera¬ 
tor.  This  is  required  to  pro¬ 
vide  consistent  operation. 

o  EDITOR/FULL  SCREEN  EDITOR 

Editors  are  used  in  developing 
software,  test  drivers,  etc. 
These  tools  provide  flexiblity, 
via  English  language  commands, 
while  protecting  all  fields  used 
by  CC/M  software. 

o  FILE  COMPARISON 

Allows  automated  generation  of 
the  CC/M  required  SCR  by  com¬ 
paring  configuration  managed 
code  to  new  code  on  a  lii.e  by 
line  basis.  The  printed  SCR 
conforms  with  the  specification 
required  format. 

o  SPECIAL  UTILITIES 

Single  commands  to  accomplish 
repetitive  functions  (e.g.,  file 
copy,  list,  transfer,  etc.). 
A1  so  provides  processors  to 
inspect  definitions  of  para¬ 
meters  and  symbol  edit. 

o  HELP 


3.  IMPACT  OF  THE  SDF 

Before  the  SDF  was  developed,  many  tasks 
were  performed  on  the  deliverable  computa¬ 
tion  equipment  in  a  serial  manner  during 
software  development  and  test.  Now  these 
same  tasks  may  Le  accomplished  in  parallel 
on  the  SDF.  This  factor  alone  allows  more 
online  time  for  those  tasks  which  cannot 
be  performed  at  the  SDF  such  as  hardware- 
software  integration.  Figure  3  shows  the 
serial  versus  para? lei  work  flow. 


Information  capanility,  at  the  Figure  3  Serial  Versus  Parallel  Work  Flow 
terminal,  to  aid  users  in  proper 

SDF  utilization.  3.1  LOAD  GENERATION 


O  MATH  MODEL  TEST  (MMT) 

A  comprehensive  software  metho¬ 
dology  that  leads  to  consistent 
test  and  verification  of  design 
logic  in  a  non-real-time  mode  of 
operation.  Capable  of  exercis¬ 
ing  single  modules,  components, 
etc.,  up  to  an  entire  simulator 
load  Capable  of  accepting 
commands  in  either  tf  inter¬ 
active  or  command  file  mode. 
Software  Quality  Assurance 
requirements  (e.g.,  test  cover¬ 
age  reports,  regression  testing, 
etc.)  are  incorporated. 

o  PLOT  PACKAGES 

Allows  offline  software  and  MMT 
to  interface  with  various  output 
devices . 

o  rILE  TRANSFER 

Allows  multiple  computers  to 
transfer  files,  in  either  direc¬ 
tion,  via  RS-232  channels. 
Error  checking/verification  is 
provided. 


A  simulator  load  may  be  define-/  as  the 
complete  set  of  executable  software  and 
data  bases  required  to  simulate  the  real 
world  device,  ready  for  insertion  into  the 
computer’s  memory  system.  Prior  to  the 
SDF,  simulation  loads  were  normally  gener¬ 
ated  once  a  day.  Because  of  the  serial 
development,  and  limited  availab.'e  com¬ 
puter  time  for  detailed  testing,  subtle 
problems  existed  with  at  least  some  of  the 
newly  incorporated  softv’are.  Isolation 
and  resolution  of  these  problems  reduced 
achievable  progress  in  a  given  period  of 
time.  Furthermore,  load  generation  and 
initial  checkout,  i.e..  a  cycling  load, 
required  an  average  of  2  to  3  hours. 
Figure  4  shows  the  process  and  results. 

In  the  SDF  environment  simultaneous  execu¬ 
tion  of  tasks  provides  more  time  for  each 
engineer  to  fully  verify  his  module(?). 
Added  testing,  using  MMT,  yields  software 
wnich  is  much  more  error  free,  thus  re¬ 
ducing  the  load  generation  and  checkout 
time  to  an  average  of  20  to  30  minutes. 

Assuming  one  load  per  day,  the  use  of  a 
SDF  provides  an  online  savings  of  1.67  to 
2.5  hours  per  day,  or  from  61.8  to  77.5 
hours  per  month  (31  days). 
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Figure  4  Comparison  of  Old  and  New  Methodology  On  Load  Generation  Process 


3.2  AIRCRAFT  SYSTEMS 

Aerodynamic  and  power  plant  (engine) 
simulation  requires  complex  offline  soft¬ 
ware  to  verify  that  the  simulator  design 
and  implementation  truly  match  the  air¬ 
craft  characteristics  and  performance  as 
defined  in  the  data  available.  The  off¬ 
line  software  also  is  used  to  develop 
usable  test  criteria  in  accordance  with 
the  specification.  Before  SDF,  an  offline 
computer  complex  was  used  to  perform  these 
functions.  However,  the  ever-changing 
online  computer  vendors  soon  made  it 
necessary  to  develop  two  identical  sets  of 
aerodynamic  and  engine  modules;  one  in  the 
language  applicable  to  the  online  computer 
and  the  second  in  the  language  of  the 
offline  computer. 

Use  of  the  SDF  allows  significant  savings, 
such  as  the  following; 

o  Acutal  simulation  modules  are 
used  in  the  SDF. 

Savings:  No  duplicate  modules 
written  for  different 
computers;  no  dupli¬ 
cate  CC/M  costs. 

o  Complex  Test  Drivers  may  be  used 
across  project  boundaries. 


Savings:  No  duplicate  modules 
or  added  CC/M;  sophis¬ 
ticated  drivers  used 
on  C-130,  B-52  simula¬ 
tors  . 

o  Math  Model  Test  (MMT )  verifies 
design,  implementation,  inter¬ 
faces,  modifications,  etc. 

Saving:  More  effective  than 

traditional  online 
debug;  shortens  online 
test  phase. 

o  Auto-Test  Guide,  an  online  test 
tools,  is  fully  compatible  with 
MMT,  allowing  identical  checkout 
on  the  simulator  and  in  the  SDF. 

Savings:  Reduced  online  regres¬ 
sion  testing  associ¬ 
ated  with  discrepancy 
clearance. 

The  SDF  saved  two  weeks  of  online  HSI  time 
on  the  C-130  Program.  The  original  HSI 
schedule  provided  three  weeks  for  aero  and 
engine  integration,  while  the  actual 
integration  was  completed  in  one  week. 
Reason:  More  controlled  development 

atmosphere,  software  quality  controls,  and 
in-depth  testing  on  the  SDF. 
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3.3  CONFIGURATION  CONTROL  AND  MANAGEMENT  4.  DEPS  CAPABILITIES 


The  SDF  incorporates  many  features  which 
assure  that  CC/M  is  correctly  accomp¬ 
lished. 

o  Fully  controlled  access  to  all 

levels  of  software. 

o  Added  controls  limit  controlled 

software  updates. 

o  Log  of  all  users,  the  functions 
performed,  date,  time,  computer 
resources  utilized,  etc. 

o  Automated  revision  logging  on 

target  module  and  in  CC/M  data¬ 
base  . 

o  Automated  SCR  generation,  log 

and  reporting. 

In  addition  to  automated  capabilities, 
manual  intervention  features  have  been 
incorporated  into  the  SDF,  and  are  used  by 
CC/M  personnel  on  projects  such  as  the 
3-52  WST.  Example  of  these  functions, 
relative  to  incorporation  of  a  B-52  WST 

change,  are: 

a.  Determination  of  applicability, 

i . e . ,  All  B-52s,  B-52G  only, 

B-52H  only,  etc. 

b.  Determination  if  a  basic  (used 
simultaneously  in  more  than  one 
station)  module. 

c.  Determination  if  the  change  must 
be  retrofitted  to  previously 
delivered  simulators. 

d.  Verification  that  all  applicable 
documents  have  been  updated  and 
that  copies  are  in  the  change 
package . 

e.  Verification  that  software 
quality  requirements  have  been 
met . 

When  these  actions  a^e  completed,  the 
complete  package  is  presented  to  the 

Software  Change  Control  Board  for  approval 
or  disapproval.  Approved  changes  are 
logged  into  the  CC/M  system  and  then  sent 
to  the  Software  Controller ( s )  for  incor¬ 
poration  into  the  load. 

Historical  data,  current  status,  module 
effectivitv  listings,  software  trees  and 
reports  on  all  activities  are  available 
from  the  SDF  upon  request. 

Cost  savings  (avoidance)  has  been  esti¬ 
mated  at  $20,000  per  month  by  using  the 
SDF  for  CC/M.  This  figure  represents  our 
estimates  of  additional  personnel  required 
in  lieu  of  the  SDF  for  CC/M. 


Many  current  contracts  include  an  offline 
computer  complex  designed  to  support  the 
simulator  at  the  customer's  installation. 
The  similarity  of  mission  in  the  SDF  and 
DEI’S  indicates  a  potential  for  transfer  of 
SDF  software  to  the  DEPS. 

5  CONCLUSIONS 

Usefulness  of  the  SDF  has  been  proven  at 
Link.  A  recent  U.S.  Air  Force  simulator 
contract  had  a  program  plan,  developed 
prior  to  the  SDF,  which  contained  a  six 
month  Hardware-Software  Integration  (HSI) 
schedule.  While  the  contract  was  in 
progress  the  SDF  became  operational  and 
was  fully  used  to  control,  develop  and 
test  software.  The  effect  of  the  SDF  is 
shown  in  Figure  5. 


Figure  5  SDF  Effect  Upon  Major  Simulator 

During  the  software  test  phase  (pre-HSI ) 
the  SDF  recorded  3400  hours  of  use.  Since 
the  types  of  testing  being  done  on  the  SDF 
were  identical  to  the  testing  normally 
done  online  as  pre-HSI  test,  one  may 
conclude  that  an  hour  on  the  SDF  saved  an 
houi  on  the  simulator.  Further,  assuming 
that  the  simulator  was  available  for  test 
16  hours  per  day  and  31  days  per  month, 
the  3400  hours  translates  to  6  months  of 
simulator  time. 

HSI  on  this  simulator  progressed  unusually 
well  with  very  few  module  interface  pro¬ 
blems.  This  resulted  in  the  HSI  phase 
being  completed  in  about  one-half  of  the 
time  original iy  scheduled.  Each  simulator 
manufacturer  and  customer  can  calculate 
their  estimated  savings  when  the  schedule 
is  reduced  by  2.8  months. 


What  will  the  future  reveal?  Certainly  Will  today's  SDF  and  tools  meet  future 

more  complex  and  difficult  requirements  nee  \s?  Not  totally,  but  proven  SDF  philo- 

and  specifications,  especially  in  the  sc  .ies,  designs,  procedures,  tools,  etc. 

areas  of  verification  and  validation.  v..l  give  the  SDF  owner  a  competitive  edge 
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ABSTRACT 


A  dynamic  target  simulation  system  design  is  described  which  uses  the  real  world  as  background 
scene.  An  instructor  driven  threat  scenario  is  projected  on  a  beamsplitter  combiner.  A  collimating 
l^ns  system  makes  the  target  appear  as  part  of  the  real  world  viewed  tnrough  the  beamsplitter.  High 
resolution  target  signatures  are  obtained  by  slirinking  525  raster  lines  down  to  a  minimum  area.  A  360 
degree  visual  hemisphere  is  provided  by  synchronization  of  gunner’ i  line  of  sight  and  location  of  the 
simulated  target  in  space.  Tne  gunner’s  line  of  sight  is  fed  to  the  microcomputer  control  via  an 
electromagnetic  sensor.  An  occupation  system  is  provided  for  the  target  to  disappear  as  it  moves 
behind  trees  or  mountains.  Scoring  capability  is  also  provided,  and  events  may  be  stored  and  played 
back  for  training/review  at  a  later  date.  Since  the  system  is  portable  and  relatively  inexpensive,  it 
will  readily  lend  itself  as  a  visual  target  simulator  for  outdoor  field  training  in  antiarmor  (e.g., 

DRAGON  and  TOW)  end  air  defense  weapons  (e.g.,  STINGER  and  CHAP  ^RRALL, 

INTRODUCTION  visual  simulator  is  designed  so  that  it  can  be  hooked  up 

to  the  actual  weapons  or  existing  outdoor  weapon 

Field  training  in  target  acquisition  currently  simulators.  This  allows  maximum  transfer  of  training 

requires  provision  of  live  aircraft/tanks/armored  to  the  real  weapons  at  considerably  reduced  cost, 

vehicles  or  live  targets  to  represent  threat  scenarios. 

Considering  aircraft/target  availability,  rising  cost  of  Provisions  for  Crew  Training 

fuel,  restricted  flight-pat'is/range-ureas,  high  cost  of 

live  targets  end  their  limited  performance  envelopes,  The  visual  simulator  is  designed  such  that  the 

it  is  not  surprising  that  most  gunner  training  is  system  parameters  may  be  viewed/varied  not  j:ily  by 

currently  perform  d  indoors  using  projection  systems  the  gunner,  but  by  the  crew  in  which  the  gunner  is  a 

and  domes  (1)  or  simply  a  low  cost  arcade  game  in  an  member.  In  this  manner,  the  system  allows  simulation 

air  conditioned  classroom  (2).  of  crew  interaction,  particularly  among  smaller  units. 


Obviously  we  cannot  expect  an  indoor  trained 
gunner  with  little  or  no  training  in  adverse  field  con¬ 
ditions  to  confidently  engage  a  high-value  threat  on 
the  first  attempt  in  an  actual  battlefield.  To  provide 
maximum  operational  proficiency  and  confidence  in 
firing  the  actual  weapon  in  adverse  environmental 
conditions,  field  training  is  an  absolute  necessity  (3). 

The  purpose  of  this  paper  is  to  present  a  cost 
effective  visual  system  to  simulate  realistic  moving 
targets  for  outdoor  gunner  training. 

DESIGN  CRITERIA 

The  proposed  concept  was  formulated  based  on  the 
following  criteria  and  guidelines  recently  outlined  by 
several  high  ranking  officers  of  military  training 
commands  (4). 


Dollar  Leverage 

Training  exercises  using  ‘  irgeLs  are  very  costly 
and  are  also  limited  by  the  size  oi  range  areas, 
target/aircraft  availability,  restricted  flight  paths  and 
friendliness  of  our  own  aircrafts.  Tne  cost  factor  is 
especially  important  because  long  and  frequent  repeti¬ 
tion  ot  exercise  is  required  to  develop  good  operator 
proficiency. 

The  proposed  visual  simulator  presents  a  low  cost 
alternative  to  simulation  dome  structures  and  their 
projection  systems.  Utilizing  the  real  world  as 
background  scene,  there  is  no  need  to  use  expensive 
••'omputer  image  generators  (CIG)  to  create  realistic 
visual  scene  details.  High  resolution  targets  are 
generated  through  utilizatior  of  relatively  low  cos: 
microprocessors,  video  discs  and  optical  components. 


Commonality  of  Device 


The  system  is  designed  so  that  it  car,  be  used  in 
conjunction  with  several  surface-to-surface  or 
surface-to-air  weapons.  Examples  are  STINGER, 
CHAPARRAL,  VULCAN,  UIVAD,  DRAGON,  TOW, 
REDEYE  and  almost  any  other  weapon  system  that 
requires  visual  acquisition  and  tracking  of  moving 
targets. 


Actual  Weapons  for  Training 


The  rising  cost  of  simulation,  and  therefore  the 
need  to  trade-off  simulators  for  the  weapon  system 
itself,  has  resulted  in  emerging  preference  to  use, 
whenever  possible,  the  actual  weapon  system  with 
dummy  rounds  as  a  training  device.  The  proposed 


Consequently,  it  is  believed  that  the  proposed 
device  offers  a  real  dollar  leverage  pay  off  to  the  user 
community. 


SYSTEM  DESCRIPTION 


System  Overview 


Figure  1  shows  a  schematic  diagram  of  the  sys¬ 
tem.  Prerecorded  moving  target  scenarios  are  gener¬ 
ated  by  mircropFoc-vssor-driven  video  discs.  A  position 
sensor  is  used  to  provide  gun-pointing  direction  or  the 
gunner’s  line  of  sight  in  pitch,  roll  and  yaw.  This 
information  is  synchronized  with  corresponding  frames 
on  video  disc  to  generate  appropriate  visual  cues. 
Frame  identification  numbers  and  Society  of  Motion 
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TRAINEE  PERFORMANCE  TO  INSTRUCTOR 
MONITOR  AND  RECORDER 


Figure  1.  System  Schematic 


Picture  and  Television  Engineers  (S VI PTE)  codes  are 
used  not  only  for  proper  sequencing  of  frames  but  to 
also  contain  target  information  such  as  range,  direc¬ 
tion,  speed  and  its  center  of  gravity  (CG)  within  each 
frame.  The  microprocessor  uses  the  gunner  interface 
inputs,  his  aim  point  and  the  target  information  to 
generate  tracking  and  scoring  data  for  evaluation  of 
gunner  performance. 

To  better  illustrate  the  system,  we  describe  its 
application  to  STINGER  training.  STINGER  is  an  air 
defense  system  capable  of  being  shoulder  fired  and 
destroying  high  value  targets  ranging  from  hovering 
helicopters  to  high  speed  maneuvering  aircraft  (5). 
Presently,  live  lire  training  for  STINGER  gunners  is 
accomplished  using  the  Stinger  Launch  Simulato- 
(STLS)  against  live  aircraft  as  shown  in  Figure  2.  The 
training  tasks  and  decisions  are  shown  in  Figure  3.  The 
initial  tasks  of  scanning  the  sky  and  detecting  the 
target  is  generally  achieved  by  an  assistant  gunner  who 
uses  both  eyes  and  points  to  the  target  as  shown  in 
Figure  4.  The  gunner  then  shoulders  the  weapon7 
simulator  and  performs  the  remaining  tasks  using 
monocular  v:sion. 

An  artist's  concept  of  the  proposed  visual  system 
applied  to  outdoor  Stinger  Training  is  shown  in  Fig¬ 
ure  5.  Note  that  the  gunner  and  his  assistant  share  the 
hardware/software  used  to  measure  the  line  of  sight 
coordinates  and  \o  generate  and  synchronize  their 
corresponding  visual  cues. 


Off-Line  Moving  Target  Scene  Generation 

Simulation  of  moving  targets  can  be  recorded 
remotely  using  computer  graphics/animation 
techniques  or  by  utilizing  miniature  models  of 
potential  enemy  aircrafts  against  a  plain  back  drop  to 
generate  realistic  target  maneuvers.  Using  either 
method,  a  video  tape  of  several  threat  scenarios  is 
obtained.  For  each  video  frame,  he  coordinates  of 
the  target  in  space,  its  range,  sp<  ed  and  the  coor¬ 
dinates  of  the  picture  element  (pixel)  representing  the 
CG  of  the  target  or  its  IR  plume  within  the  frame  are 
determined.  Using  either  the  SMPTE  or  Manchester 
Codes,  the  location,  range  and  the  CG  information  is 
stored  w’thin  each  frame  during  tape-to-disc 
conversion.  The  purpose  of  storing  this  information  is 
to  reduce  software  overhead  involved  in  scanning  each 
frame  when  gunner's  line  of  sight  is  sensed  to  position 
the  appropriate  frame  against  the  real  world.  The 
information  is  also  used  for  real  time  generation  of 
guiuier’s  tracking  data. 

Several  off-the-shelf  industrial  video  disc  units 
are  described  in  Reference  6.  Most  units  provide 
54,000  frames  allowing  up  to  30  minutes  recording  of 
threat  scenarios.  The  instructor  may  choose  any  pre¬ 
recorded  scenario  and  make  it  visible  to  the  trainee  or 
observer  through  the  Target  Display  System.  To  help 
the  trainee  find  the  target,  an  audio  simulator  gener¬ 
ates  the  target  noise  signature  from  the  direction  of 
the  target. 
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Figure  2.  Outdoor  Stinger  Training  Against  Live  Aircraft 


1 

* 


NOTE 


FINAL  DECISION  TO  FIRE 
IS  MADE  AT  THIS  POINT 


Figure  3.  Stinger  Training  Tasks  and  Decisions 
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Figure  4.  Crew  Interaction  of  Stinger  Trainee  and  Assistant  ('.tinner 
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Figure  5.  The  Visual  Simulation  Concept  Applied  to  Outdoor  Stinger  Training 


The  Target  Display  System  can  be  either  monoc¬ 
ular  or  binocular  depending  on  the  weapon  system  and 
the  training  requirements  established  during  front-end 
analysis.  A  helmet  mounted  binocular  system  would  be 
appropriate  for  Chaparral  training  whereas  a  weaoor 
mounted  monocular  system  may  be  used  for  TOW 
training.  As  for  Stinger,  both  types  can  be  used  —  a 
weapon  mounted  display  (WMD)  for  the  gunner,  and  a 
helmet  mourned  display  (HMD)  for  the  assistant  gunner 
as  shown  in  Figure  f.  HMD  and  WMD  have  the  same 
basic  design.  The  former  uses  2  CRTs  whereas  the 
lattei  uses  only  one.  The  microcomputer  controlled 
input  froi  i  video  disc  is  fed  to  e  high  resolution,  high 
brightness  miniature  CRT  such  as  Ferranti’s  Microspot 
02 13/ 117  weighing  less  than  3.2  oz  with  volume  below 
4  cubic  incites.  A  lightweight  optical  system  of 
mirrors,  collimating  lens  and  beamsplitter  is  used  to 
project  the  moving  target  to  infinity  so  that  it  appears 
as  part  of  the  real  world. 


A  schematic  of  a  helmet  mounted  target  display 
system  is  shown  in  Figure  6.  The  parameters  that 
should  be  considered  include  size,  weight,  exit  pupil, 
eye  relief,  field  of  view  (FOV),  distortion,  beamsplitter 


reflectivity/transmissivity,  image  to  ghost  ratio  and 
several  others.  These  parameters  and  their  values  for 
several  off-the-shelf  HMD's  are  described  in 
Reference  7. 

Since  the  beamsplitter  can  have  a  neutral  density 
coating  with  at  least  75%  transmission  (=25% 
reflectivity),  the  brightness  reduction  of  the  external 
scene  is  hardly  noticeable.  Assuming  the  optical 
elements  used  to  project  the  CRT  image  pass  about 
85%  of  the  light,  the  effective  transmission  for  the 
moving  target  is  therefore  21%  (=.85  x  .25). 

Accordingly  a  5000  ft  lambert  brightness  target  will 
appear  at  5000  x  .21  =  1050  ft.  lambert  to  the  eye. 

This  provides  sufficient  contrast  to  allow  visibility 
against  a  10.000  lambert  sky  background. 

The  HMD  shown  in  Figure  6  has  the  capability  of 
achieving  very  high  resolution  over  the  Tub  sphere  of 
vision  because  the  generated  moving  target  fills  only 
the  instantaneous  FOV  of  the  observer.  The  instan¬ 
taneous  FOV  is  100°  wide  by  60°  high.  Each  eye  is 
furnished  with  a  60°  circular  by  60®  vertical  FOV.  The 
overlap  field  between  the  eyes  is  20'-.  As  for  the 
WMD,  the  gunner’s  FOV  is  limited  only  by  the 
structure  of  the  weapon  sight  as  shown  in  Figure  7. 


Figure  6.  Helmet  Mounted  Target  Display  System 
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Figure  7.  Stinger  Gunner’s  View  Through  the  Weapon  Sight 


Target  Oceultation 

Figure  8  illustrates  the  observer's  view  through 
his  HMD  beamsplitter,  Note  that  the  far  target  is 
occulted  by  a  mountain.  This  dynamic  concealment  is 
accomplisheu  bv  the  oceultation  unit  shown  in  Fig¬ 
ure  9.  The  three  35mm  films  are  driven  together  by 
the  motor-tach  in  response  to  the  observers  azimuth 
rotation  of  his  head  or  weapon.  In  this  manner  the 
film  silhouettes  are  always  aligned  to  the  real  world 
background.  The  films  are  made  in  the  laboratory  to 
provide  only  black  and  white  information  from 
panoramic  pnotographs  taken  from  l Ik?  trainees 
station.  This  assures  exact  correspondence  between 
sillKJucttes  and  background  ridges  separat  ng  sky  from 
terrain  A  maximum  of  three  ridges  will  t»e  used  since 
it  is  felt  that  this  will  provide  a  sufficient  number  of 
valleys  within  which  enemy  aircraft  can  approach  with 
concealment.  The  television  camera  may  bo  made  to 
"see"  any  of  the  three  ridges  by  merely  indexing  the 
film  drive  system  up  or  down.  This  ridge  data  permits 
the  oceultation  of  the  aircraft  by  terrain  f  atures  so 
tlmt  flight  behind  mountains  and  reappearance  is  easily 
accomplished  in  accordance  with  the  principles 
outlined  in  'deference  8. 


In  operation  >e  azimuth  rotation  of  the 
observer's  helmet  weapon  rotates  the  film  unit  so  that 
the  TV  camera  always  "looks"  *t  the  same  real  world 
silhouette  as  the  observer  is  viewing.  When  f  r  ’*»nis  to 
the  aircraft  location  lie  will  see  the  aircraft  **iLin  hi* 
field  of  view.  This  requires  not  only  the  correct 
azimuthal  direction  but  also  hi*  elevation  angle  of 
view  and  his  head  roll  angle  to  present  the  proper 
altitude  through  his  sight.  These  angles  are  generated 
by  a  line  of  sight  sensor. 


Line  of  Sight  Sensor 

The  threat  scenario  should  be  seen  only  when  the 
observer  looks  in  the  direction  of  the  moving  target 
location.  This  requires  a  high  precision  sensing  device 
that  continually  supplies  the  host  microcomputer  with 
elevation,  roll  and  azimuth  of  the  gunner's  weapon 
and  or  the  observer’s  helmet  19).  Such  sensors  are 
called  Helmet  Mounted  Piek-offs  (MMP).  The 
parameters  that  should  be  considered  include  size, 
weight,  accuracy,  angular  coverage,  freedom  of  move¬ 
ment,  slew  rate,  update  rate  and  the  cost  effective¬ 
ness  of  the  sensor.  There  are  several  o.f-the-shelf 
HMPs  developed  using  mechanical,  light  emitting 
diode*,  ultra-sonic  or  infrared-based  sensing 
techniques. 

V  candidate  JIMP  for  the  visual  simulator  i>  an 
electromagnetic  system  developed  by  PoTncmus 
Navigation  Sciences  as  shown  in  Figure  10  The 
principal  of  operation  of  this  dvviee  is  described  ;n 
Reference  10.  Feasibility  experiments  utilizing  the 
Pol  he  nr  is  sensor  in  conjunction  with  HMD's,  have  been 
conducted  by  the  Aerospace  Medical  Research  Labor¬ 
atory  at  W  right  Patterson  A 1  H  and  the  Advanced 
Simulation  concepts  Laboratory  at  N  TKC  111).  These 
experiments  indicate  that  the  magnetic  sensor's 
attitude  precision  exceeds  0.025°  and  its  lliroughput 
delay  is  around  10  milliseconds.  T!»e  angle  output  i* 
updated  6u  lures  per  »vCond.  Since  the  video  stream 
runs  at  30  frames  per  second,  this  allows  over  la 
millisecond*  of  microprocessing  time  between 
eon*eeutivc  frames. 

One  advantage  of  the  Pot  hem  us  device  is  that  it 
eau  handle  two  sensor*  simultaneously.  Applied  to 
Stinger  Training,  this  would  generate  line  of  sight 
information  for  both  the  gunner  and  his  assistant,  thus 
allowing  simulation  of  crew  interaction  wlule  sharing 
the  same  hardware  software  configuration. 
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Figure  10.  Major  Components  of  an  Electromagnetic  Helmet- Mounted  Piek-off 


Microcomputer  System  and  Software  Design 

The  nucleus  of  the  system  consists  of  a  16-Oit 
CFU,  operating  system  ROMs  and  scratch  pad  KAMs. 
Four  off-the-shelf  microprocessor  families  have  been 
evaluated.  These  are  Digital  Equipment  Corporation, 
LSI  11/23.  Motorola  6#00U.  Intel  8086  and  Zilog  8000. 
The  68000  family  appears  to  be  most  suited  for  this 
application  due  to  iLs  short  I/O  interrupt  kernel 
execution  time  (33  ms  versus  114  ms  for  DEC'S  LSI 
11/23),  and  its  relatively  fast  bit  set.  reset  test 
requiring  least  bytes  of  code. 

To  maintain  commonality  of  the  device  across 
several  weapon  systems,  the  operating  system  contains 
firmware  designed  to  perform  generic  control  func¬ 
tions  such  as  synchromzatioi  of  HM1*  sensor  updates 
and  moving  target  image  selection  from  vuko  disc, 
tracking  and  scoring  algorithms,  and  synchronization 
of  audio  and  video  signatures  based  on  gunner’s  line  of 
sight.  Weapon  dependent  control  functions  such  as 
gunner’s  interface  data  are  stored  in  a  KOM  for  a 
particular  weapon  system.  The  HUM  is  mterchai.ge- 
able  from  one  weapon  to  another.  In  this  manner,  the 
bulk  of  system  software  is  made  independent  of  a 
particular  weapon,  thus  providing  the  ability  to 
optimally  tailor  the  visual  simulator  to  the 
requirements  of  a  specific  weapon  system. 

Scoring  Capability 

As  part  of  the  V\  Ml*  module,  a  Cl  I  V  is  located 
on  the  weapon  and  boresigtited  to  the  gunner's  sight. 


As  the  gunner  tracks,  the  instructor  secs  the  target 
moving  against  the  background  on  his  C1U  monitor. 
For  Stinger,  the  gunner’s  direction  of  look,  supplied  by 
a  weapon-mounted  sensor,  is  indicated  by  a  15° 
(relative  to  the  real  world)  circular  reticle.  His  cl  fort 
m  tracking  the  target  becomes  obvious  ana  his 
electrically  sensed  trigger  pull  freezes  H  e  circular 
reticle  and  the  CKT  scene.  This  permits  scoring  of 
his  accuracy  as  well  as  acquisition  time. 

lor  wire-guided  weapon  systems  such  as  TOW  or 
DKACON,  the  gunner’s  aiming  error  is  ea  culated 
every  33  milliseconds  using  real  time  HMP  input  and 
the  moving  target’s  IK  pixel  coordinates  from  each 
video  frame.  The  aiming  error  is  then  used  oy  the  host 
microcomputer  to  generate  guidance  error  commands 
and  to  analyze  gunner's  performance. 


CONCLUSION 

Fhe  design  concepts  discussed  in  tins  paper 
represents  an  effort  by  the  training  di  vice  industry  to 
be  resf?onsive  to  interserviee  training  requirements,  in 
terms  of  realism,  the  visual  system  described  offers 
the  3-dimensu>nal  sights  and  sound;  of  the  real  world. 
Doing  microprocessor-based  and  using  off-the-shelf 
equiprties..,  the  system  is  extremely  cost  effective. 

The  commonality  of  hardware  components  and 
software  architecture  make  tin*  device  applicable  to 
several  weapon  systems. 
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THE  TRENT)  TOWARDS  AREA  OF  INTEREST  IN  VISUAL  SIMULATION  TECHNOLOGY 


Simulation  of  the  view  seen  by  the  pilot 
through  the  windows  of  a  cockpit  has  been  one 
of  the  most  difficult  problems  in  simulation 
technology.  For  takeoff  and  landing  with 
fixed  wing  aircraft,  computer  generated 
imagery  displayed  on  one  or  more  cathode  ray 
tubes,  each  fitted  with  infinity  image  optics, 
has  reached  a  satisfactory  stage  of  develop¬ 
ment.  and  is  now  a  very  widely  used  technique. 
For  air  combat  maneuvering  (ACM),  satisfactory 
results  have  been  achieved  on  a  few  trainers 
over  a  period  of  more  than  two  decades  by  pro¬ 
jecting  an  Image  of  a  target  aircraft  onto  the 
inside  of  a  spherical  screen  surrounding  the 
simulator  cockpit.  However,  the  simulation  of 
a  wide  field  of  view  of  terrain  in  high  detail 
as  Is  needed  in  military  simulation  for  low 
altitude  flight,  navigation,  target  acquisi¬ 
tion,  weapon  delivery,  threat  avoidance  and 
confined  area  maneuvering  for  both  fixed  wing 
aircraft  and  helicopters  remains  a  difficult 
and  expensive  problem.  It  Is  the  "last 
frontier”*  in  visual  simulation. 

The  best  currently  available  approach  to 
providing  the  required  high  resolution,  wide 
field  of  view  (FOV)  display  is  to  divide  the 
FOV  between  a  number  of  butted  displays  sur¬ 
rounding  the  pilot.  Each  display  requires  its 
own  channel  of  computer  image  generation 
(CTG,.  The  number  of  displays  and  CIC  chan¬ 
nels  required  depends  on  the  total  displayed 
FOV  required,  the  resolution  required  and  the 
number  of  picture  elements  (pixels)  that  can 
be  displayed  in  each  window.  Television  sys¬ 
tems  with  1023  scan  lines  are  becoming  more 
common  (as  compared  with  the  broadcast  system 
standard  of  525  lines)  so  that  approximately 
one  million  pixels  per  display  are  available. 
To  cover  a  hemisphere  with  imagery  wfthjresol- 
ution  of  2  arc  minutes  per  pixel  (a  typical 
requirement),  approximately  24  displays  and  24 
GIG  channels  would  be  needed.  This  is  not 
practicable  on  the  grounds  of  acquisition  cost 
alone;  the  system  would  also  pose  problems  in 

*  with  acknowledgements  to  J.  H.  Bums 


wide  field  of  view,  high  detail  density 
being  answered  by  developments  In  both  the 
inefficiency  and  cost  of  filling  the  large 
butted  images,  various  techniques  are  being 
of  interest  (AOI)  which  is  usually  of  high 
Xh-is  paper  reviews  the  advan- 
that  is  tracked  with  a  target, 
made  to  recent  Navy,  Air  Force 


petting  up  and  maintenance  which  would  lead  to 
high  running  cost.  These  practical  considera¬ 
tions  restrict  a  multiple  projector  system  to 
five  to  eight  channels,  each  covering  about 
70°  and  giving  a  resolution  of  6-9  arc  minutes 
per  pixel.  The  cost  of  such  a  system  Is  still 
approximately  $20M. 

As  an  alternative  to  generating  and  dis¬ 
playing  imagery  over  the  full  FOV  required  by 
the  pilot  for  carrying  out  the  necessary 
maneuvers,  Imagery  can  be  concentrated  in 
those  directions  that  are  most  useful  to  him, 
resulting  in  significant  cost  savings  in  both 
image  generation  and  display  hardware.  This 
general  concept  is  almost  as  old  as  visual 
simulation  itself  (one  example  being  in  ACM 
simulation  as  already  described)  but  it  can  be 
implemented  in  many  different  ways  and  new  ap¬ 
proaches  are  being  developed.  It  is  time  that 
there  different  approaches  were  compared  and 
their  limitations  and  advantages  discussed. 
In  the  rest  of  this  paper,  any  display  in 
which  at  least  part  of  the  FOV  is  not  fixed  in 
direction  relative  to  the  aircraft  windows 
will  be  referred  to  as  an  area  of  interest 
(AOI)  display. 

TYPES  OF  AOI  DISPLAY 

The  movement  of  the  FOV  with  respect  to 
the  aircraft  windows  to  create  an  AOI  display 
can  be  controlled  in  various  ways.  The  FOV 
may  move  with  or  track: 

a.  a  displayed  target 

b.  the  pilot's  head 

c.  the  pilot’s  eyes 

d.  a  combination  of  the  above. 

Most  of  the  systems  have  two  fields  of 
view,  one  set  In9ide  the  other.  To  provide  a 
common  terminology,  the  larger  FOV  will  be 
called  the  main  FOV  and  the  smaller  the  inset 
FOV. 

Target  Tracked  Displays;  Air-to-Air 

In  a  target  tracked  system,  the  inset  FOV 
is  placed  dynamically  within  the  main  FOV  «c- 
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ABSTRACT 


"Vhe  challenge  of  providing  a  cost  and  training  effective 
visual  environment  to  the  trainee  in  a  flight  simulator  is 
generation  and  display  of  visual  imagery.  To  overcome  the 
field  of  view  using  multiple  television  projectors  giving 
developed  for  concentrating  high  Image  detail  in  an  area 
resolution  and  set  within  a  larger  field -of  view  of  low  resolution., 
tapes  and  disadvantages  of  the  various  AOI  techniques  including  an  AOI 
tracked  with  head  attitude,  and  eye  tracked.  Particular  reference  is 
and  Army  developments*, 
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cording  to  the  computed  position  of  the  target 
with  respect  to  the  pilot's  own  aircraft. 
Most  applications  to  date  have  been  of  the 
type  described  for  ACM  and  gunnery  where  the 
target  is  another  aircraft  displayed  on  the 
inside  of  a  spherical  screen  in  high  detail 
over  an  inset  FOV  of  about  15°,  using  a  servo 
directed  television  projector.  The  main  FOV 
is  provided  typically  by  a  wide  angle  gimha.ed 
shadowgraph  projector  giving  a  low  resolution, 
dim  image  of  the  horizon,  sky  and  a  suggestion 
of  the  terrain  so  that  attitude  (but  not 
translation)  cues  are  available. 

The  target  aircraft  image  is  superimposed 
on  the  sky  background  and  always  appears 
brighter  than  the  sky;  this  is  not  usually  in 
accordance  with  the  real  world,  but  is  gen¬ 

erally  considered  acceptable.  Such  a  system 
is  efficient  in  that  high  detail  in  the  main 

FOV  is  not  needed  and  is  not  provided.  Im¬ 
proved  aerodynamic  simulation  has  led  to  sev¬ 
eral  systems  of  this  type  being  brought  into 

use  recently,  to  give  effective  training  in 
high  altitude  ACM. 

In  the  Navy,  systems  of  this  type  include 
Device  2E6,  the  Air  Combat  Maneuvering  trainer 
for  the  F-4  and  F-14  and  Device  2  F 1 12,  two 
F-14  Weapon  System  Trainers  (WST).  The  AOI 
image  is  generated  using  a  television  camera 
viewing  a  plane  model  mounted  on  gimbals. 

The  Navy's  Visual  Technology  Research  Sim¬ 
ulator  (VTRS)  has  demonstrated  the  feasibility 
of  applying  CIG  to  an  AOI  display  coupled  with 
a  background  display  on  a  spherical  screen  for 
military  applications.  Military  tasks  which 
have  been  demonstrated  includ-  carrier  land¬ 
ing,  formation  and  tactical  .*rmation  flight, 
gunnery,  air-to-ground  weapon  delivery  and 
hostile  environment  maneuvering.  Trainers  in 
devel<  nt  which  will  apply  this  visual  tech¬ 
nology  -icept  include  the  Navy's  F-18  Weapon 
Tactics  Trainer  (WTT)  and  the  Marine  Corps 
AV-8B  WTT.  Examples  of  other  target  tracked 
displays  include  an  early  Air  Force  lew  cost 
formation  flight  trainer,  which  presented  a 
90°  FOV  of  another  aircraft,  and  the  Northrop 
LASWAVS  which  presents  a  60  FOV  from  a  tele¬ 
vision  camera  viewing  a  modelboard. 


Target  Tracked  Displays:  Ai r-to-Cround 


Where  a  target  trucked  inset  is  set  a- 
gainst  a  main  FOV  the  requirements  are  not 
particularly  stringent,  as  long  as  the  main 
FOV  is  relatively  featureless,  such  as  sky  (in 
the  case  of  a  target  aircraft),  or  sea  (in  the 
case  of  carrier  landing).  However,  where  air- 
to-ground  tasks  must  be  simulated,  the  ground 
target  area,  including  the  terrain  immediately 
surrounding  the  target  itself,  is  displayed  as 
ar.  inset  at  higher  resolution  than  the  main 
FOV  (using  either  modelboard/televi sion  camera 
or  CIG  image  generators),  and  the  system  re¬ 
quirements  become  more  stringent  in  three  re¬ 
spects.  First,  the  computation  and  inset  pro¬ 
jector  servo  requirements  are  more  exacting  as 
the  inset  image  has  to  register  with  the  main 


image  during  all  maneuver.*:,  whereas  air  tar¬ 
gets  do  not  have  a  visual  reference  and  small 
positioning  errors  are  not  seen.  Second,  the 
distortion  of  the  image  constituting  the  main 
FOV  must  be  minimized  so  that  when  the  inset 
takes  up  its  correct  position,  the  background 
imagery  is  also  correct.  Third,  the  node  of 
insetting  needs  considerat ion  because  st'aighl- 
forward  superimposition  (as  for  air/air  sys¬ 
tems)  may  not  be  fully  acceptable  and  it  may 
be  necessary  to  "cut  a  hole"  in  the  background 
image  to  make  way  for  the  inset.  These  re¬ 
quirements  are  considered  further  in  a  later 
sect  ion . 


Before  discussing  specific  head  and  eye 
tracked  systems,  some  background  will  be  given 
on  the  relevant  human  characteristics. 
Figure  l(J)  shows  the  visual  field  available 
for  a  given  head  pointing  direction:  binocu¬ 
lar  vision  extends  horizontally  to  +  70°  and 
vertically  to  +  50°,  -  70°;  monocular  vision 
adds  another  30°  horizontally  each 


FIGURE  1  ■  MONOCULAR  AND  BINOCULAR  VISUAL  FIELDS 

side.  Figure  2(--)  gives  the  distribution  of 
visual  acuity  across  the  retina,  from  which  it 
can  be  seen  that  for  an  eye  fixated  on  the 
center  of  a  40°  diameter  spot  (and  resolving  1 
arc  minute  at  the  center),  the  resolution  at 
the  edges  of  the  spot  will  be  onlv  about  10 
arc  minutes.  Figure  3^^  is  a  collation  of 
data  from  several  sources  on  the  range  and 
velocity  cf  head  and  eye  movements  encountered 
for  various  tasks.  From  the  information  in 
the  three  figures,  it  is  possible  to  postulate 
variejs  displa*’  systems  that  take  advantage  of 
human  head  and  eye  characteristics  for  various 
maneuvers. 
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FIGURE  2  •  DISTRIBUTION  OF  VISUAL  ACUITY  ACROSS  THE 
RETINA  EXPRESSED  IN  DEGREES  FROM 
THE  FOVEA 

Head  Tracked  Displays 

Consider  first  a  system  wi  th  head  track  inf* 
only,  i.e.,  the  displayed  image  is  moved  so  as 
to  keep  its  centroid  always  in  line  with  the 
head  pointing  direction  by  monitoring  the  head 
attitude  continuously  with  a  head  tracker  and 
commanding  the  CIG  image  generator  to  compute 
its  image  with  the  apprppriate  viewing  direc¬ 
tion.  With  the  eyes  pointing  straight  ahead, 
a  displayed  F OV  of  140°  horizontally  and  120° 
vertically  would  provide  all  necessary  visual 
cues  except  for  the  peripheral  monocular  parts 
of  the  FOV.  Deflection  of  the  eyes  by  20° 
(more  than  20°  only  occurs  4%  of  the  time) 
adds  to  these  figures  to  give  180°  horizontal¬ 
ly  and  160°  vertically.  So  it  may  be  said 
that  If  head  tracking  is  adopted,  there  is  no 
point  in  exceeding  an  FOV  of  180°  x  160°. 

Actual,  figures  for  specific  systems  will, 
of  course,  depend  on  the  training  task  and  the 
cost  effectiveness  of  providing  as  large  an 
FOV  as  this.  A  head  tracked  display  without 
any  limitation  in  following  head  pointing  dir¬ 
ection,  but  with  a  small  FOV  of  soy  50°  hori¬ 
zontally  (a  single  fixed  display  of  50°  hori¬ 
zontally  by  36°  vertically  has  been  used  for 
many  years  by  the  airlines  for  landing  and 
takeoff)  will  enable  some  maneuvers  to  be  car¬ 
ried  out  normally.  For  other  maneuvers  (see 
"Intense  Visual  Search"  in  Figure  3),  an  un¬ 
natural  amount  of  head  movement  will  be  re¬ 
quired  and  it  may  not  be  possible  to  car*-y  out 
the  task  correctly.  In  any  event,  from  some 
experimental  work  carried  out  at  NAVTRAEQUIP- 
CKN,  a  small  head  tracked  FOV  is  made  much 
more  acceptable  if  the  edges  are  softened, 
i.e.,  blended  to  black.  Blending  to  midgrey 
is  even  better  in  removing  the  obtrusi veneos 
of  the  edges  of  the  FOV. 


A  very  important  consideration  with  head 
tracked  displays  is  the  CIC  throughput  delay. 
Assuming  the  head  tracker  response  time  is 
negligible,  a  change  in  head  attitude  will 
cause  a  demand  for  a  new  view  and  it  is  essen¬ 
tial  that  the  image  presented  to  the  pilot 
should  not  move  in  its  apparent  direction  in 
space  during  this  period,  causing  unnatural 
"swimming”  of  the  image.  Once  the  new  image 
has  been  computed,  it  must  be  displayed  in  the 
correct  direction.  If  the  head  tracked  FOV  is 
obtained  from  a  television  projector  mounted 
in  a  fixed  relationship  to  the  cockpit  struc¬ 
ture,  the  displayed  format  direction  may  lag 
the  head  direction,  but  displayed  objects  must 
remain  fixed  in  relation  to  the  screen.  For 
systems  in  which  the  display,  such  as  one  us¬ 
ing  cathode  ray  tubes,  is  mounted  on  the  head, 
the  displayed  format  moves  with  the  head  but 
compensation  of  image  position  is  required  to 
avoid  "swimming."  This  point  will  be  elabor¬ 
ated  later  when  different  systems  under  devel¬ 
opment  are  reviewed. 

One  interesting  possibility  with  head 
tracked  displays,  if  head  position  in  relation 
to  the  cockpit  is  tracked  as  well  as  head  at¬ 
titude,  is  to  obtain  some  of  the  effects  of  a 
collimated  display  in  those  systems  where  the 
image  is  viewed  on  a  screen.  Movements  of  the 
head  are  measured  and  fed  to  the  CIC  image 
generator  to  give  a  corresponding  change  in 
the  computed  viewpoint.  The  effect,  parti¬ 
cularly  for  sideways  movements  of  the  head,  is 
that  the  objects  viewed  stay  fixed  in  space 
with  head  movement,  and  this  is  a  strong  cue 
to  the  distance  of  objects.  The  results  are 
similar  to  what  is  obtained  with  a  collimated 
display.  CIG  throughput  delay  may  be  a  prob¬ 
lem  with  rapid  head  movements. 

Eye  Tracked  Displays 

Let  us  now  turn  to  the  consideration  of 
eye-tracked  displays  in  which  the  eye  pointing 
direction  of  the  pilot  1b  monitored  continu¬ 
ously  and  the  CIG  generates  the  visual  infor¬ 
mation  such  that  it  is  always  concentrated  in 
the  eye  pointing  direction.  From  Figure  2,  if 
a  single  field  of  view  display  were  to  be  pre¬ 
sented  to  the  pilot  in  which  the  resolution 
decreased  smoothly  away  from  the  center  of 
vision,  the  central  resolution  would  be  avail¬ 
able  in  all  directions  in  which  he  could  look, 
but  the  total  amount  of  information  required 
to  be  displayed  would — be  very  greatly  de¬ 
creased  as  compared  with  a  system  having 
everywhere  a  resolution  equal  to  the  central 
resolution.  Smoothly  varying  resolution  is 
difficult  to  implement  but  an  approximation  to 
the  curve  of  Figure  2  can  be  made  by  having  a 
high  resolution  inset  FOV  to  provide  for  fo- 
veal  vision  inside  a  main  FOV  of  lower  resolu¬ 
tion,  with  the  main  FOV  image  removed  over  the 
area  of  the  inset. 

In  this  context,  it  Is  necessary  to  dis¬ 
tinguish  between  resolution  and  detail,  where 
detail  may  be  defined  as  density  of  CIG  edges 
or  average  number  of  edges  per  unit  solid 
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angle  of  view.  CIG  data  bases  are  modelled 
with  a  number  of  levels  of  detail,  e.g.,  a 
house  at  the  lowest  level  of  detail  is  just  a 
block  (sufficiently  detailed  at  a  large  range) 
but  at  a  higher  level  it  h^s  doors  and  windows 
(necessary  for  close  range) .  Art  inset  may 
have  either  higher  resolution  or  higher  level 
of  detail  (LOD)  than  the  main  FOV,  or  both. 
For  eye  tracked  systems,  high  resolution  in 
the  inset  is  necessary  and  it  is  uneconomic  to 
provide  it  in  the  main  FOV  also  where  the  eye 
cannot  resolve  it.  Since  the  inset  occupies 
only  a  small  FOV,  the  edge  density  can  be  much 
higher  than  in  the  main  FOV,  although  the  in¬ 
set  CIG  channel  does  not  display  more  edges 
than  the  main  FOV.  This  higher  edge  density 
requires  a  higher  LOD  from  the  data  base  ard  a 
higher  display  resolution.  Thus,  we  arrive  at 
a  preferred  system  in  which  the  inset  has  both 
higher  resolution  and  higher  LOD. 

An  important  consideration  in  eye  tracked 
systems  5s  the  boundary  between  the  inset  and 
the  main  FOV.  A  sharp  edge  is  highly  undesir¬ 
able  and  a  blend  between  the  two  over  part  of 
the  inset  area  is  necessary  to  avoid  visibili¬ 
ty  of  the  boundary. 

Another  important  consideration,  as  for 
head  tracked  displays,  is  the  throughput  delay 
of  the  CIG  imago  generator.  When  the  eye  com¬ 
mences  a  rapid  movement  from  one  fixation  dir¬ 
ection  to  another,  i.e.,  it  commences  a  ar¬ 
cade,  the  eye  tracker  commands  the  CIG  to 
generate  a  view  appropriate  to  the  new  viewing 
direction.  The  time  taken  by  the  CIG  to  do 
this  is  c.round  80  msec  and  the  system  must  be 
such  tha':  the  inset  is  not  visible  until  the 
image  correct  for  the  new  direction  is  avail¬ 
able.  This  means  that  the  eve,  if  it  moves 
fast  enough,  will  be  looking  at  part  of  the 
low  resolution  main  FOV  for  some  milliseconds 
before  the  high  resolution  of  the  inset  ap¬ 
pears.  At  NAVTRAEQUIPCEN,  it  was  considered 
necessary  to  carry  out  an  experiment  to  obtain 
some  practical  data  on  acceptable  CIG  time  de¬ 
lay.  Other  factors,  referred  to  above,  which 
needed  evaluation  were  the  acceptable  size  for 
the  inset  and  the  width  of  the  blend  region  in 
an  eye  tracked  display. 

Experiments  were  performed  at  NAVTRAFQl’IP- 
CEN'**'  In  which  images  were  projected  from  a 
special  slide  projector  to  cover  a  spherically 
shaped  screen  surrounding  the  subject.  A  var¬ 
iable  resolution  mask,  overlaying  the  slide, 
modified  the  image  such  that  it  had  a  central 
high  resolution  area  surrounded  by  a  low  re¬ 
solution  area  with  a  blend  region  between 
then.  An  eye  tracker,  using  infrared  light, 
measured  the  azimuthal  pointing  direction  ot 
one  of  the  subject  's  eyes  and  drove  a  rapid 
servomotor  attached  to  the  mask.  The  subject, 
therefore,  saw  a  high  detail  image  in  the  cen¬ 
ter  of  his  vision  at  all  times.  A  variable 
time  delay  could  be  inserted  between  the  eye 
tracker  and  the  servo. 

Various  masks  were  used  having  different 
widths  of  blend  region;  a  very  small  or  non¬ 


existent  blend  region  was  found  to  be  highly 
objectionable  and  distracting  to  most  observ¬ 
ers.  The  experiments  Indicated  that  an  inset 
width  of  25°  within  which  there  is  a  5°  wide 
smoothly  varying  transition  region  combined 
with  a  delay  of  80  msec  and  an  eye  tracker  ac¬ 
curacy  of  +  2.5°  would  cause  noticeable,  but 
not  objectionable,  perception  of  the  borders 
of  the  inset. 

This  experiment  does  not  provide  exact 
simulation  of  the  appearance  of  a  working  eye- 
tracked  visual  simulation  display,  although  it 
gives  useful  guidelines  on  the  design  of  such 
a  system.  In  particular,  it  does  not  simulate 
the  appearance  of  the  different  levels  of  de¬ 
tail  of  a  CIG  system;  this  will  be  discussed 
ir.  a  later  section  of  this  paper. 

AOI  DISPLAYS  IN  DEVELOPMENT 

To  advance  the  consideration  of  AOI  dis¬ 
plays,  it  is  necessary  to  refer  to  systems 
currently  in  development.  Table  I  lists  the 
various  Government  funded  systems.  Systems  1 
and  2  have  a  target  tracked  inset  FOV  in  a 
fixed  main  FOV  for  air-to-ground  use;  System  3 
has  a  single  head  tracked  FOV;  System  A  has  a 
head  tracked  main  FOV  with  a  head  tracked  in¬ 
set;  System  5  has  a  fixed  main  FOV  with  an  eye 
tracked  inset;  and  System  6  has  a  head  tracked 
main  FOV  with  an  eye  tracked  inset.  In  refer¬ 
ring  to  Table  1  and  the  following  description, 
it  has  to  he  realized  that  these  systems  are 
in  various  stages  of  development  and  the  per¬ 
formance  data  given  are  target  figures  for 
feasibility  models  only.  Not  all  these  sys¬ 
tems  will  be  developed  to  procurement  of  a 
full',  engineered  prototype,  but  a  comparison 
of  them  in  terms  of  fundamental  advantages  and 
limitations  will  be  valuable  in  understanding 
the  potential  gains  with  AOI  displays.  All 
use  displays  with  1023  TV  line  capability. 

System  1  in  Table  1  refers  to  experimental 
work  with  VTRS  at  NAVTRAEQUI  PCEN .  The  main 
FOV  provides  a  low  detail  view  of  terrain  from 
the  background  projector  while  the  target  pro¬ 
jector  provides  an  inset  FOV  of  the  target 
area  with  higher  resolution.  (Higher  LOD  for 
the  inset  FOV  is  in  the  process  of  being  im¬ 
plemented.)  The  inset  can  show  a  group  ot 
,  lildings  and  part  oi  a  road  or  a  group  ot 
ta.<ks.  Various  maneuvers  can  be  carried  out, 
including  strafing  the  target,  without  losing 
registration  between  the  low  kind  high  resolu¬ 
tion  images. 

To  achieve  this  degree  of  dynamic  »«gis- 
t rat  ion,  the  servo  response  ot  the  target  pro¬ 
jector  had  to  be  optimized.  Its  pointing  ac¬ 
curacy  was  ♦  1  arc  minute  under  static  condi¬ 
tions  and  lrat  !>0'/sec. 

Another  essential  factor,  permitting  reg¬ 
istration  of  the  inset  image  with  the  main 
image,  is  correction  ot  the  d’stortion  of  the 
main  image  on  the  spherical  screen.  This  dis¬ 
tortion  arises  due  to  noncoinc  Jdence  of  I  he 
pilot's  eyes  and  the  projector  exit  pupil,  the 
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display  optics  and  oth?r  factors.  The  correc¬ 
tion  is  done  in  the  C1G  image  generator^) 
by  breaking  up  long  CIG  edges  into  shorter 
edges  and  repositioning  the  vertices  to  map 
the  scene  onto  the  screen  such  that  the  dis¬ 
tortion  is  less  than  0.1%  from  the  pilot's  eye 
position.  For  the  inset,  the  smaller  FOV 
makes  distortion  correction  less  critical,  but 
this  is  being  implemented. 

The  method  of  adding  •'he  inset  to  the  main 
FOV  is  simply  to  overlay  ..^e  inset  image  on 
the  background  image,  as  for  tl  3  target  plane 
in  air-to-air  simulators.  This  m^_''s  that  the 
inset  has  to  be  brighter  than  the  uackground 
and  in  fact  appears  as  a  fairly  well  defined 
bright  disc.  Then  is,  therefore,  no  question 
of  the  pilot  having  to  search  for  the  target 
as  in  the  real  world;  not  only  is  it  rendered 
in  higher  detail,  but  it  is  also  brighter. 
However,  ot.ce  the  pilot  has  acquired  the  tar¬ 
get,  the  maneuvers  he  carries  out  should  not 
be  affected  by  its  somewhat  artificial  appear¬ 
ance.  F.  tperimental  work  with  pilots  using  the 
technique  described  will  take  place  nex'  year 
at  VTRf. 

System  2  in  Table  I  is  the  High  Resolution 
Area  (HRA)  Dual  Projector  Display  system^) 
iunded  by  the  Army  (Project  Manager,  Training 
Devices)  and  procured  by  the  Human  Resources 
Laboratory  at  Williams  AFB.  The  Advanced  Sim¬ 
ulator  for  Pilot  Training  (ASPT)  at  Williams 
AFB  has  a  visual  system  consisting  of  seven 
*'acets  of  a  dodecahedron  structure  to  provide 
a  wide  FOV  display.  Each  facet  contains  a 
Farrand  Pancake  Window  optical  system  and 
utilizes  a  large  3h  inch  diameter  cathode  ray 
tube  as  a  display  source.  The  optical  ele¬ 
ments  of  the  Pancake  Window  produce  an  image 
of  the  CRT  face  at  infinity. 

The  Dual  Projector  Display  is  an  experi¬ 
mental  replacement  of  the  CRT  by  two  1023  line- 
color  television  light  valve  projectors  fitted 
with  optics  to  project  images  onto  a  back  pro¬ 
jector  screen  of  the  size  and  shape  of  the  CRT 
faceplate.  One  projector  provides  the  main 
(70°)  FOV  covering  the  whole  screen  and  the 
other,  together  with  a  servo  driven  mirror, 
gives  an  approximately  10°  inset  of  high  reso¬ 
lution.  An  important  feature  of  this  experi¬ 
mental  system  is  the  capability  of  (a)  demon¬ 
strating  the  removal  of  the  main  image  over 
the  area  ot  the  inset  (thus  "cutting  a  help" 
to  leave  room  for  the  inset)  and  (b)  demon¬ 
strating  the  effect  of  various  blending  func¬ 
tions  for  the  region  around  the  inset.  This 
is  a  further  development  beyond  simply  super¬ 
imposing  the  inset  as  has  been  demonstrated  on 
VTRS . 

The  system  had  reached  a  certain  stage  of 
development  in  January  19d2,  and  the  inset 
could  be  moved  around  within  the  main  FOV.  A 
band  of  blending  between  the  inset  and  the 
main  FOV  was  generated  by  varying  the  gain  of 
the  video  signals  from  the  two  projectors  in  a 
complementary  manner,  using  eight  steps  of 
gain.  This  demonstration  showed  that  more 


steps,  or  a  smooth  gain  change,  would  be  nec¬ 
essary  to  avoid  high  visibility  in  the  blend 
region  and  also  shewed  the  sensitivity  o i  the 
system  to  misalignment  between  inset  aid  main 
FOV  and  to  the  variation  of  color  which  occurs 
across  a  light  valve  display  (causing  color 
mismatch  for  some  positions  of  the  inset). 
The  experiment  was  valuable  in  its  Impact  on 
future  AOI  systems. 

System  3  of  Table  I  is  the  Visually  Cou¬ 
pled  Airborne  System  Simulator  (VCASS)  which 
has  been  the  subject  of  research  by  the  Air 
Force  Aerospace  Medical  Research  Laboratory 
(AMRL)  for  many  years.  The  primary  purpose 
for  its  development  is  for  aircraft  display 
hardware  and  crew  station  configuration  devel¬ 
opment.  However,  VCASS  represents  one  of  the 
important  alternatives  in  the  range  of  possi¬ 
ble  techniques  for  providing  a  pilot  in  a  sim¬ 
ulator  with  a  view  In  any  direction  and  so 
must  be  included  here. 

The  principle  of  VCASS,  as  used  for  simu¬ 
lation,  is  to  mount  on  the  pilot's  helmet  two 
small  CRTj  on  which  CIG  imagery  representing 
the  real  world  is  displayed,  am.  to  present 
this  imagery  to  the  pilot  using  a  miniature 
Farrand  Jancake  Window  for  each  eye.  Each 
Pancake  Window  presents  an  image  of  the  cor¬ 
responding  CRT  face  at  inf ii. tty  over  an  FOV  of 
80°  horizontaMy  by  60°  vertically.  These 
fields  can  be  overlapped  to  give  a  total  hori¬ 
zontal  FOV  of  between  140°  and  100°  (with  cor¬ 
responding  horizontal  overlap  of  between  20° 
ana  o.>°).  The  system  is  head  tracked,  i.e., 
it  uses  a  Polhemus  Head  Attitude  Sensor,  which 
uses  a  fixed  magnetic  field  radiator,  detected 
by  a  sensor  on  the  pilot's  helmet,  to  generate 
data  representing  the  roll,  pitch  and  yaw  dir¬ 
ections  and  translational  position  of  the 
pilot's  head.  The  CIG  image  generator  is  con¬ 
trolled  by  this  data  to  produce  the  appro¬ 
priate  view  for  the  instantaneous  head  atti¬ 
tude  so  that  correct  Imagery  is  available  at 
all  times.  To  avoid  the  imagery  representing 
the  outside  world  appearing  not  only  through 
the  windows,  but  also  superimposet  on  the  in¬ 
strument  panel,  a  CIG  model  of  the  aircraft 
windows  can  be  mapped  Into  the  viewing  plane 
(the  CRT  faces)  and  used  to  blank  out  the 
image  except  where  it  occupies  the  window 
area,  for  the  pilot's  instantaneous  head  posi¬ 
tion  and  attitude. 

The  Pancak;  Windows  have  approximately  7% 
transmission  for  the  direct  view  of  the  inter¬ 
ior  of  the  cockpit  over  a  somewhat  smaller  FOV 
than  the  display.  The  CIC  system  currently  in 
use  is  calligraphic. 

As  noted  earlier,  a  head  mounted  AOI  dis¬ 
play  has  the  characteristic  that  the  slightest 
attitude  change  of  the  pilot's  head  immediate¬ 
ly  gives  a  corresponding  change  in  t he  direc- 
Cioi  in  space  in  which  the  imagery  is  seen. 
The  CIG  responds  to  the  head  tracker  in  com¬ 
puting  the  new  view,  but  there  is  a  momentary 
mica!  ignme.'t  due  to  the  throughput  delay.  Re- 
fcivKttf  *.  he  head  velocity  figures  in  Figure 


3  and  ignoring  the  highest  velocities  since 
vision  deteriorates  beyond  about  30°/sec,  a 
minimum  figure  of  30°/sec  may  be  taken.  For  a 
throughput  delay  of  80  msec,  the  corresponding 
momentary  angular  error  in  the  display  is  2.4 
degrees.  Experimental  work  at  NAVTRAEQUIPCEN 
(to  be  described  under  System  6  of  Table  I) 
has  shown  that  the  subjective  effect  of  the 
resultant  swimming  of  the  image  in  a  head 
mounted  CRT  display  is  disturbing  and  some 
form  of  correction  should  be  applied.  It  is 
understood  that  improved  head  attitude  sensing 
algorithms  may  be  developed  later  for  VCASS 
and  no  doubt  this  problem  would  be  addressed 
at  that  time. 

The  concept  of  presenting  an  image  separ¬ 
ately  to  each  eye  in  a  helmet  mounted  config¬ 
uration  has  important  consequences.  First  of 
all,  it  makes  possible  a  two-pilot  system  in 
which  each  of  two  p..  lots  can  be  given  his  own 
independent  view  of  the  world  (requiring  a 
fair  amount  of  duplication  in  the  image  gener¬ 
ator).  Second,  U  makes  stereo  viewing  possi¬ 
ble,  which  is  being  investigated  on  VCASS. 
However,  a  nrice  must  be  paid  in  terms  of  com¬ 
plication:  two  CRTs  and  two  sets  of  viewing 

optics  are  needed  and  this  increases  the 
weight.  Furthermore,  fairly  exacting  adjust¬ 
ments  are  needed  to  set  up  the  display  for  any 
given  pilot,  in  terms  of  interpupilary  dis¬ 
tance,  shape  of  the  pilot's  head,  etc.  The 
concept  is  being  further  evolved  and  miniature 
color  CRTs  may  be  developed  and  a  raster  scan 
CIO  may  be  used  later  to  provide  the  imagery. 

The  VCASS  system  does  not  have  a  high  res¬ 
olution  inset  FOV;  this  is  precluded  by  the 
resolution  that  can  bo  obtained  from  a  minia¬ 
ture  CRT.  An  inset  FOV  is  provided  in  the 
second  Air  Force  system,  described  below,  at 
the  expense  of  greater  complexity. 

System  4,  the  Combat  Mission  Trainer  (CMT) 
is  being  developed  for  the  Human  Resources 
Laboratory  at  Williams  AFB  by  CAE  of  Canada. 
The  display  optics  are  the  same  ns  for  the 
AMRL  system  (miniature  Pancake  Windows)  but 
four  light  valve  television  projectors  are 
used  instead  of  helmet  mounted  CRTs,  the 
images  being  relayed  to  the  helmet  through 
coherent  fiber  optic  guides.  Two  projectors 
provide  the  main  FOVs  for  the  two  eyes  and  a 
further  pair,  each  with  its  own  fiber  optic 
guide,  provides  a  slewable  inset  rOV  to  each 
eye . 

The  potential  performance  of  this  system 
compared  with  VCASS  is  greater,  in  that  the 
high  resolution  inset  (at  present  head 
tracked,  but  possibly,  in  the  future,  eye 
tracked)  extends  the  possible  range  of  use. 
Whether  the  performance  can  be  realized  de¬ 
pends  on  some  difficult  optical  design  prob¬ 
lems.  The  problems  center  on  the  fiber  optic 
guides  and  associated  optics  to  couple  them  to 
the  projectors;  it  is  difficult  to  make  coher¬ 
ent  fiber  optic  guides  of  several  million  in¬ 
dividual  fibers  without  some  broken  fibers, 
giving  lack  spots  on  the  display,  and  such 


guides  are  fairly  Inflexible.  By  contrast, 
the  VCASS  system  has  only  lightweight  flexible 
cables  connected  to  the  helmet.  The  CMT  has 
the  same  capability  for  two-pilot  display  and 
for  stereo  viewing.  Color  Is,  of  course, 
readily  available  by  using  color  projectors. 
Other  comments  made  about  the  VCASS  System  are 
applicable  on  the  problem  of  CIG  throughput 
delay,  the  need  to  blank  out  imagery  falling 
on  the  inside  of  the  cockpit  and  the  advan¬ 
tages  and  problems  of  presenting  images  separ¬ 
ately  to  each  eye. 

System  5  of  Table  I  Is  the  Eye-slaved  Dis¬ 
play  Integration  and  Test  (EDIT)  system.^) 
The  original  concept  was  proposed  and  partial-- 
ly  implemented  by  Singer  Link  for  the  Air 
Force  Aeronautical  Systems  Division,  Deputy 
for  Simulators  (ASD/YW),  Project  2360,  which 
included  an  advanced  visual  system  for  A-10 
and  F-16  training.  The  project  was  termin¬ 
ated,  but  some  hardware  and  software  became 
available  for  experimental  use  and  further 
work  is  in  progress  by  Singer  Link,  funded 
jointly  by  ASD/YW  and  NAVTRAEQUIPCEN,  to  com¬ 
plete  the  key  components  of  the  system  and 
then  integrate  them  with  VTRS  for  test  and 
evaluation.  The  data  shown  in  Table  I  relates 
to  this  work  and  not  to  the  original  2360  spe¬ 
cif  icat ion. 

The  concept  calls  for  a  fixed  main  FOV 
from  one  light  valve  projector  (in  a  simulator 
for  training  the  main  FOV  could  be  made  to 
cover  as  large  an  FOV  as  desired  by  using  sev¬ 
eral  projectors)  together  with  a  "foveal  pro¬ 
jector4’  capable  of  rapid  slewing  in  accordance 
with  output  data  from  an  eye  tracker  and  pro¬ 
viding  a  small,  eye  tracked  inset  FOV.  The 
foveal  projector  is  mounted  rigidly  in  rela¬ 
tion  to  the  cockpit  structure  ip  contrast  to 
the  helmet  mounted  arrangement  for  providing 
the  inset  FOV  with  System  4  (CMT). 

This  has  several  consequences.  First, 
movement  of  the  pilot's  head  does  not  automa¬ 
tically  cause  movement  of  the  projected  inset 
image  with  relation  to  the  screen  as  is  the 
case  where  the  display  is  actually  mounted  on 
the  helmet.  The  movement  of  the  inset  is  con¬ 
trolled  by  the  head  tracker  measuring  head  at¬ 
titude  change  with  respect  to  the  cockpit  and 
the  eve  tracker  measuring  eye  attitude  change 
with  respect  to  the  head,  the  two  streams  of 
data  being  combined  to  command  the  foveal  pro¬ 
jector  servos  to  take  up  the  new  pointing  dir¬ 
ection.  There  is,  therefore,  no  need  to  com¬ 
pensate  for  CIC  throughput  delay  as  far  as 
head  attitude  is  concerned.  Second,  t*>e  servo 
response  mast  be  extremely  rapid  to  come  near 
to  matching  eye  movement  rates  (see  Figure 
3).  Third,  because  the  pilot's  head  and  the 
foveal  projector  exit  pup* 1  are  considerably 
separated,  distortion  of  the  inset  image  oc¬ 
curs  and  varies  with  position  in  the  main  FOV, 
which  must  be  compensated,  and  the  throw  dis¬ 
tance  varies  requiring  servo  control  of  the 
projector  lens  focus. 


Finally,  to  add  the  inset  to  the  main 
image,  a  hole  is  cut  electronically  in  the 
main  imago  and  duo  to  the  variation  of  distor¬ 
tion  with  position,  the  hole  shape  has  to  be 
dynamically  varied.  A  smooth  blend  is  pro¬ 
vided  around  the  inset.  Some  indication  has 
already  been  given  of  the  possible  problem  in 
presenting  the  new  inset  image  following  a 
saccade,  due  to  CIG  throughput  delay.  Experi¬ 
ments  carried  out  by  Singer  Link  indicate  that 
the  phenomenon  known  as  saccadic  suppression, 
which  causes  the  eye  to  be  insensitive  for 
some  tens  of  milliseconds  following  a  saccade, 
will  allow  time  for  the  new  image  to  be  gener- 
a  ted . 

The  EDIT  project  is  a  very  interesting  one 
ns  it  aims  at  the  greatest  efficiency  in  gen¬ 
erating  and  displaying  imagery  by  employing 
eve  tracking.  Integration  of  the  system  into 
VTRS  followed  by  pilot  testing  is  at  present 
planned  to  commence  early  in  1984. 

The  final  system  listed  in  Table  I,  Sys¬ 
tem  6,  Laser  Helmet  Mounted  Di splay ( ® ) , 
has  been  developed  at  NAVTRAEQUIPCEN  and  a 
complete  system  is  now  being  procured.  Both 
head  and  eye  tracking  are  used,  the  inset  FOV 
being  fixed  in  the  center  of  the  main  FOV  and 
the  resulting  combined  FOV  directed  to  follow 
the  eye  direction  in  space.  The  source  of 
light  to  generate  the  image  is  a  laser  system 
giving  red,  green  and  blue  primary  colors  and 
the  display  is  viewed  on  a  retroref lect i ve 
spherical  screen.  The  light  is  modulated  by 
the  video  signals  from  the  CIG  and  scanned  in 
a  line  by  a  rotating  mirror  polygon.  Three 
frame  scanners  are  mounted  on  the  helmet,  on-; 
for  the  main  FOV,  one  for  the  inset,  and  one 
for  offsets  In  the  line  scan  direction,  each 
giving  a  1023  line  raster.  Fiber  optic  rib¬ 
bons  are  used  to  transmit  the  light  to  the 

helmet;  these  are  light  and  flexible  compared 
with  the  full  frame  fiber  optic  guides  re¬ 

quired  for  System  4  (CMT). 

The  use  of  laser  light,  scanned  opto¬ 
mechanical  ly  ,  has  Interesting  implications.  A 
system  of  this  kind,  unlike  a  light  valve  pro¬ 
jector,  can  exhibit  absolute  uniformity  in  the 
intensity  of  the  projected  beam  over  the  FOV 
and  there  do  not  appear  to  be  any  serious 
problems  in  matching  the  color  and  luminance 
of  the  inset  image  to  the  main  FOV  Image. 

To  prove  the  concept  as  far  as  possible 
prior  to  procurement,  a  mockup  was  built  omit¬ 
ting  the  eye  tracked  inset  and  using  a  1023 

line  CIG  signal,  to  give  a  head  tracked  FOV  of 
approximately  25°  on  a  3  ft.  radius  spherical 
retroref lecting  screen.  Compensation  for 
throughput  delay  was  demonstrated,  using  the 
VTRS  CIG  image  generator,  by  momentary  deflec¬ 
tion  of  the  raster  using  offset  signals  to  the 
line  and  frame  scanners  computed  from  the  dif¬ 
ference  between  current  head  attitude  in  pitch 
and  yaw  and  the  pitch  and  yaw  attitudes  used 
by  the  CIG  to  compute  the  current  scene.  The 
results  of  the  experiment  gave  confidence  that 
n  helmet  mounted  laser  display  was  feasible; 


in  particular,  CIG  throughput  delay  was  suc¬ 
cessfully  compensated  giving  stable  imagery. 
The  fiber  optic  ribbons  of  1000  fibers,  made 
to  NAVTRAEQUIPCEN  speci ficatlon,  have  not  yet 
been  satisfactory  as  to  the  presence  of  broken 
or  distorted  fibers.  A  ribbon  Is,  however, 
much  easier  to  make  than  a  full  frame  guide. 

As  far  as  the  eye  tracked  inset  is  con¬ 
cerned,  the  work  previously  discussed^) 
gave  confidence  that,  given  quick  response 
from  the  eye  tracker,  an  acceptable  result 
would  be  obtained.  The  NAVTRAEQUIPCEN  laser 
HMD  system  is  about  to  be  procured.  The  plan 
calls  for  integration  with  VTRS  commencing 
part  way  through  FY  85  followed  by  human  fac¬ 
tors  evaluation  during  FY  86. 

API  Blending  and  ''Popping" 

Before  attempting  to  sum  up  as  to  the  ad¬ 
vantages  and  disadvantages  of  the  various  A0I 
systems,  the  question  of  how  well  the  inset 
merges  with  the  main  FOV  for  the  systems  that 
have  an  inset  should  be  discussed.  First, 
there  ie  the  question  of  whether  the  inset  is 
simply  junerimposed  on  the  main  FOV  Image  or  a 
hole  is  cut  in  the  main  FOV  image  and  the  In¬ 
set  Inserted.  Experimental  evidence  to  date 
favors  cutting  the  hole  and  inserting  the  in¬ 
set  provided  a  blend  region  giving  a  smooth 
change  between  the  two  regions  is  used.  How¬ 
ever,  this  has  not  yet  been  Implemented  in  a 
working  prototype,  and  an  optimum  ratio  of 
width  of  blend  region  to  inset  width  needs  ad¬ 
ditional  experimentation. 

Secondly,  there  Is  the  question  of  "pop¬ 
ping.”  As  observed  previously,  an  Inset  FOV 
has  not  only  a  higher  resolution,  but  also 
should  use  a  higher  level  of  detail  (L0D)  from 
the  CIG  data  base  than  does  the  main  FOV,  so 
that,  for  example,  a  runway  may  be  featureless 
at  the  low  LOD,  but  have  stripes  at  the  higher 
LCD.  In  CIG  as  normally  implemented,  a  higher 
LOD  Is  brought  in  as  the  range  decreases  and 
thiB  can  be  done  slowly  so  that,  for  example, 
the  runway  stripes  gradually  fade  in  on  top  of 
the  previously  blank  runway.  If  the  stripes 
occur  In  an  A0I,  but  not  in  the  main  FOV, 
movement  of  the  A0I  may  cause  them  to  appear 
suddenly,  and  this  has  been  referred  to  as 
popping. 

For  a  target  tracked  system  with  a  fixed 
ground  target,  popping  cannot  occur,  although 
the  appearance  may  be  somewhat  unrealistic 
owing  to  the  target  area  standing  out  in  high¬ 
er  resolution.  For  a  target  tracked  system 
with  a  moving  target,  a  fast  target  can  cause 
popping,  to  a  degree  dependent  on  the  data 
base . 

Eye  tracked  systems,  in  which  the  inset 
moves  within  the  main  FOV,  can  exhibit  popping 
also  but  the  eye  can  never,  by  definition, 
look  directly  at  the  blend  region  and  the  eye 
is  operating  at  lower  resolution  at  the  edge 
of  the  inset.  With  head  tracked  systems,  in 
which  an  inset  is  fixed  at  the  center  of  the 
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I0V*  thc  °>'e  can  look  directly  at  the  blend 
region. 

The  popping  question  has  been  considered 
sufficiently  important  at  NTEC  to  .lead  to  a 
decision  to  carry  out  some  basic  experiments 
simulating  CIG  objects  by  sinusoidal  bar  pat¬ 
terns  of  varying  spatial  frequency  and  ampli¬ 
tude.  It  is  hoped  that  this  work,  to  be  ac¬ 
complished  during  FY  S3,  will  quantify  the 
problem  and  provide  guidelines  to  CIG  model¬ 
lers  on  minimizing  the  effect. 

THE  OUTLOOK  FOR  AOI 

The  trend  towards  AOI  displays  due  to  the 
excessive  cost  of  implementing  wide  angle  vis¬ 
ual  systems  with  multiple  projector,  multiple 
CIG  channel  techniques  is  likely  to  continue 
foi  a  good  many  years  to  come.  The  eye 
tracked  systems  offer  the  greatest  potential 
for  high  performance  to  cost  ratio,  with  a 
resolution  of  1  -  1  1/2  arc  minutes  per  pixel 
in  the  eye  pointing  direction  (and  so  effec¬ 
tively  in  any  direction)  with  the  need  for 
only  two  CIG  channels.  A  key  question  is 
whether  an  eye-directed  inset  can  appear 
natural  to  the  pilot  and  whether  he  will  be 
able  to  perform  with  such  a  system  without 
eyestrain  or  other  physiological  problems.  It 
is  certainly  to  he  hoped  that  the  funding 
identified  to  support  the  NAVTRAEQUIPCEN  HMD 
and  the  AS  D/  YW  /  NAVTRAEQUIPCEN  EDIT  remains 
available  as  these  two  systems  represent  thc 
two  main  alternatives  -  on-head  mounting  and 
ot f-head  mounting  -  and  only  properly  carried 
out  integration  and  test  will  allow  the  best 
system  to  be  chosen. 

If  eve  tracked  systems  do  not,  in  the  end, 
prove  to  be  practicable,  or  if  they  turn  out 
to  be  more  expensive  than  hoped,  a  head 
tracked  system  such  as  VCASS  may  perhaps  be 
developed  as  a  cost  effective  visual  system 
with  more  restricted,  but  useful  characteris¬ 
tics.  McDonnell  Douglas  Electronics  is  work¬ 
ing  on  a  similar  system,  using  a  VITAL  IV  cal-- 
ligraphic  CIG  night  scene  image  generator, 
giving  a  40°  circular  FOV  for  each  eye  and  a 
larger  total  field  with  partial  overlap.  Such 
systems  do  not,  of  course,  have  the  effective 
resolution  of  the  eye  tracked  systems. 

The  res-,  mtion  may  be  increased  in  the 
center  of  the  FOV  to  match  that  achievable 

with  the  eye  tracked  systems  by  using  a  system 
such  ns  the  CMT  (System  4)  in  which  both  the 
main  FOV  and  the  inset  FOV  move  with  the 

head.  With  such  a  system  it  is  necessary  to 

turn  the  head  to  bring  the  high  resolution 
area  to  bear  on  the  object  viewed,  whereas  in 
the  real  world  situation  the  eyes  move  rapidly 
to  acquire  objects  which  are  then  seen  with 

high  resolution.  It  remains  to  be  determined 
whether  such  a  different  viewing  pattern  will 
permit  satisfactory  training. 

The  target  tracked  systems,  including  the 
type  of  system  demonstrated  for  air-to-ground 
use  on  VTRS  using  a  target  projector  do  pro¬ 


vide  an  alternative  to  the  eye  tracking  sys¬ 
tems  and  certainly  systems  along  the  lines  of 
VTRS  could  be  implemented  with  little  risk. 
The  value  of  emphasizing  a  target  by  showing 
it  with  greater  resolution  will  be  explored 
next  year  on  VTRS  using  pilots  in  human  fac¬ 
tors  experiments.  We  shall  have  to  wait  three 
years  or  more  for  validation  of  the  eye 
tracked  systems. 
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THE  TECHNICAL  CONTRIBUTIONS  Of  THE 
TACTICAL  COMBAT  TRAINER  DEVELOPMENT  PROGRAM 
James  0.  Basinger  and  John  M,  Wilson, 

N  Aeronautical  Systems  Division 

\  X  Wright-Patterson  Air  Force  Base,  Ohio 

\  X.  Robert  A.  Fisher,  Singer  Company, 

_ _  \  X  Link  Division,  Binghamton,  New  York 

ABSTRACT 

Pfoject  2360^the  Tactical  Combat  Trainer  (TCT),  was  an  Air  Force  Engineering  Develop¬ 
ment  Program  to  develop  two  prototype  Weapon  System  Trainers  (WST)  for  training  A-10,  F-15, 
and  F- 1 6  pilots  for  combat.  Each  prototype  was  to  consist  of  two  visual  simulation  systems 
integrated  with  two  previously  manufactured  Operational  Flight  Trainers  (OFT)  to  form  a 
single  WST ky^he___two_  sepa rate  cockpit  stations  would  permit  two  pilots  to  Pfly^  mutual 
support  mi ssionToFa's  opponents  in  air-to-air  combat. '"The  WST.  was  designed  to  provide 
full  mission  training  fur  air-to-air  and  a i r-to-surface  combat  tasks .  Xrhe  TCT  prototypes 
were  being  developed  under  contract  by  the  General  Electric  Company  and  the  Singer  Company 
for  a  r^fly  offrto  select  a  production  contractor.  Approximately  two  years,  into  the 
program,  these  contracts  were  terminated  due  to  USAF  budget  problems..  Before  termination, 
however ,  important  studies  and  developments  were  completed  in  the  visual  simulation  area 
by  each  contractor.  Both  Singer  and  GE  proposed  an  Area-o f- Interest  (AOI)  visual  system 
and  used  Computer  Image  Generation  •  (CIG) .  The  General  Electric  approach  was  based  upon 
a  head-slaved  AOI  infinity  image  display.  The  Singer-Link  approach  was  based  upon  an  eye- 
slaved  AOI  projected  on  a  dome  % 


TCT  PROGRAM  | 

The  Tactical  Combat  Trainer  (TCT)  Droaram  was 
to  develop  a  full  mission  Weapon  System  Trainer  1 
(WST)  to  train  A-10,  F-15,  and  F-I6  pilots  in 
coinbat  skills.  The  WST  was  to  create  an  environ¬ 
ment  where  the  pilot  could  (1)  coalesce  all  his 
training  from  separate  sources,  (2)  experience  the 
heavy  task  loading  of  combat,  and  (3)  experience 
aspects  of  combat  which  cannot  be  trained  in  the 
air  in  peacetime,  such  as  SAM  launch,  acquisition, 
and  defensive  maneuvers  or  low  altitude  air-to-air 
combat.  The  full  mission  capability  of  the  WST 
was  to  include  takeoff,  formation  (close,  route, 
and  tactical),  aerial  refueling,  low  altitude 
tactical  navigation,  air-to-air  or  a i r-to-surface 
combat,  return  to  base  and  landing. 

Project  2360  was  a  head-to-head  competitive 
Engineering  Development  Program  to  develop  two 
prototype  TCT  systems,  each  by  a  different  con¬ 
tractor.  These  prototypes  were  then  to  be  evalu¬ 
ated  in  a  "fly  off"  to  select  a  production  con¬ 
tractor  to  build  an  additional  twenty-six  units. 
Each  TCT  was  to  consist  of  two  Government  Furnished 
Operational  Flight  Trainers  (OFT),  a  visual  display 
for  each  OFT,  an  image  generation  system  for  each 
display,  modified  OFT  Instructor/Operator  Stations 
(IOS)  and  modified  OFT  Electronic  Warfare  (EW) 
simulation  systems. 

The  TCT  prototypes  were  to  be  designed  to  be 
integrated  with  A-10  OFTs  manufactured  by  Reflec- 
tone.  Incorpora  ted .  The  production  WSTs  would  be 
integrated  with  F-15  and  F- 16  OFTs  manufactured  by  ■ 
the  Goodyear  Aerospace  Company  and  the  Singer 
Company,  respectively.  The  prototype  development 
contracts  consisted  primarily  of  developing  the 
visual  simulation  system,  integration  with  OFTs, 
modifying  the  Instructor/Operator  Stations  (IOS) 
and  modifying  the  Electronic  Warfare  (EW)  simula¬ 
tion  system.  The  OFTs,  along  with  their  respective 
visual  systems,  were  to  be  integrated  to  permit 
either  two  aircraft  mutual  support  missons  or  fly¬ 


ing  as  opposing  aircraft.  The  IOS  was  to  be  de¬ 
signed  for  a  single  instructor  to  monitor  and 
control  the  simulation  for  both  pilots  during 
integrated  operations.  Contracts  to  develop  the 
TCT  prototypes  were  awarded  to  the  General  Electric 
Company,  Simulation  and  Control  Systems  Depart¬ 
ment,  and  the  Singer  Company,  Link  Flight  Simula¬ 
tion  Division,  in  September 'of  1  978.  Just  prior 
to  the  completion  of  the  Critical  Design  Review 
in  December  1980,  these  contracts  were  terminated 
due  to  the  USAF  funding  problems.  The  termination 
included  residual  tasks  to  complete  selected  de¬ 
velopment  efforts  and  the  documentation  of  all 
studies  and  developments. 

The  TCT  was  designed  to  permit  combat  training 
including  evasive  flight  in  a  high  threat  (surface 
and  air)  environment;  target  detection,  recognition 
and  identification;  and  weapon  delivery  against 
either  air  or  surface  targets.  A  tactical  fighter 
pilot  primarily  relies  on  his  vision  for  target 
acquisition  and  tracking,  detecting  and  avoiding. ... 
threats,  maintaining  contact  with  his  wingman,  and 
controlling  his  own  aircraft.  Thus,  the  major 
thrust  of  the  TCT  was  to  provide  all  the  visual 
cues  that  the  pilot  needs  to  fly  combat  tasks. 

From  the  inception  of  Project  2360,  it  was 
evident  that  the  visual  display  portion  of  the 
visual  system  placed  the  highest  demand  on  tech¬ 
nology.  Due  to  the  high  risk  involved,  both  con¬ 
tractors  developed  breadboards  of  the  display 
system  to  confirm  projected  performance  and  to 
establish  subsystem  requi rements .  In  addition, 
many  tests  and  emulations  were  performed  to  es¬ 
tablish  system  performance.  Prior  to  termination, 
both  contractors  were  able  to  complete  the  devel¬ 
opment  of  specific  display  components. 

Since  Project  2360  was  a  competitive  program, 
information  concerning  the  technical  aspects  of 
the  program  was  not  published.  Due  to  the  termi¬ 
nation  of  the  contracts,  this  information  can  now 
be  presented.  This  paper  will  describe  the  TCT 
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requirements,  the  technical  approaches,  design 
performance,  and  results  of  the  development  efforts. 
Although  many  developments  and  analyses  were  com¬ 
pleted,  this  paper  will  primarily  address  the 
visual  simulation  developments  with  emphasi >  on 
the  visual  display. 

TCI  REQUIREMENTS 

The  requirements  for  the  TCT  evolved  from  the 
combined  visual  system  requirements  of  the 
Required  Operational  Capabilities  (ROCs)  for  the 
A- 1 0  (Nov  76)  and  F-15  ( J u 1  76)  and  the  require¬ 
ments  letter  for  the  F- 16  (Aug  74).  Each  of  these 
documents  required  a  full  mission  WST  which  con¬ 
sisted  to  two  OFTs  with  full  (same  as  the  air¬ 
craft)  field  of  view  ( FOV )  visual  simulation. 

Each  WST  was  to  be  capable  of  independent  (indi¬ 
vidual)  or  integrated  (mutual  support  or  opposing) 
flight  for  all  tactical  fighter  training  and 
tactical  (combat)  roles  of  the  aircraft.  These 
roles  included  takeoff  and  landing,  formation 
flight,  air  refueling,  low  altitude  tactical  navi¬ 
gation,  air-to-air  combat  and  tactical  air-to- 
surface  tasks. 

Separate  research  visual  systems  had  been 
developed  and  had  successfully  trained  air-to-air 
combat  at  medium  altitude,  takeoff  and  landing, 
and  limited  formation  tasks,  but  no  systems 
existed  to  support  low  altitude  tactical  naviga¬ 
tion,  tactical  a i r-to-surface  tasks,  or  low  alti¬ 
tude  air-to-air  combat. 

Project  2235  (PE  64227F),  the  Ai r-to-Ground 
Visual  Simulation  Demonstration,  was  initiated  by 
the  Deputy  for  Simulators  in  1975  to  evaluate 
visual  simulation  technologies  potentially  appli¬ 
cable  to  ai  r-to-surface  weapons  delivery  training. 
The  evaluation  of  three  somewhat  modified  visual 
systems  by  tactical  fighter  pilots  performing  both 
training  and  tactical  tasks  showed  that  indeed 
more  capability  existed  than  was  previously 
thought.  The  demonstration  included  both  camera 
model  and  CIG  image  generators,  optical  mosaic 
and  dome  displays,  and  the  feasibility  of  both 
head  slaved  and  target  slaved  areas  of  interest 
(AOIs).  The  demonstration  included  both  subjec¬ 
tive  evaluation  and  objective  measurement  of  the 
technical  characteristics  of  each  device.  The 
basic  recommendations  of  Project  2235  were  two¬ 
fold:  first,  begin  procurement  of  a  production 
prototype  of  a  CIG /0 p t i cal  Mosaic  visual  system, 
and  second,  pursue  research  and  development  of  a 
CIG/Dome  visual  system  including  further  defini¬ 
tion  of  area  of  interest  requi rements .  In 
addition,  the  demonstration  pointed  out  the  need 
for  further  development  to  meet  the  Tactical  Air 
Command  (TAC)  requi rements . 

Further  evaluation  of  the  technical  visual 
system  requirements  in  the  two  primary  task  areas, 
tactical  air-to-air  and  ai  r-to-surface  combat 
tasks,  revealed  that  the  requirements  were  so 
similar  that  a  single  device  was  the  preferred 
approach.  The  display  resolution  requirements 
for  air-to-air  combat  against  a  MIG-21  size  target 
are  essentially  the  same  as  the  resolution  re¬ 
quirements  for  a  tank  size  target  for  ai  r-to- 
surface  weapons  delivery.  Also,  when  the  highly 
likely  scenario  of  offensive  or  defensive  air-to- 
air  combat  at  low  altitude  is  considered  (a 
scenario  not  practiced  in  the  aircraft  for  safety 


reasons)  the  visual  cues  required  are  also  very 
similar.  All  thro'  aircraft,  the  F-15,  r-16,  and 
the  A-K)  can  expect  to  engage  or  to  be  engaged  In 
low  altitude  air-to-air  combat.  A  thorough  anal¬ 
ysis  of  training  and  tactical  (combat)  tasks  and 
the  accompanying  technical  requirements  for  a 
visual  system  strongly  support  the  development  of 
a  single  visual  system  for  training  in  both  task 
areas . 

6 

Armed  with  the  stated  user  requirements, 
demonstrated  air-to-air  simulation  capabilities, 
and  the  results  of  Project  2235;  work  was  begun 
in  1977  on  the  specification  and  statement  of  work 
for  what  became  Project  2360  (PE  64227F) ,  the 
Fi ghter/Attack  Simulator  Visual  System  (F/A5VS), 
later  known  as  the  Tactical  Combat  Trainer  (TCT) 
program. 

A  functional  specification  was  developed  for 
Project  2360  to  encourage  the  competing  contrac¬ 
tors  to  use  their  skills  and  to  be  innovative  to 
meet  the  training  requirements  of  the  user. 

Special  care  was  taken  not  to  preclude  a  partic¬ 
ular  technology  if  it  appeared  to  have  any  poten¬ 
tial  to  meet  the  training  requirements.  The  tech¬ 
nical  requirements  for  the  visual  display  speci¬ 
fied  a  minimum  FOV  and  some  limited  optical  and 
alignment  characteristics  based  upon  previous  work. 
Image  alignment  requirements  were  based  upon 
training  task  requi rements ,  the  most  stringent 
alignment  in  front  of  the  Heads  Up  Display  (HUD), 
where  it  is  more  critical.  A  CIG  system  was  re¬ 
quired  as  an  image  source  because  of  its  inherent 
flexibility  and  the  ability  to  permit  a  large 
gaming  area.  Imagery^requi rements  were  related  to 
task  requirements.  ~ 

To  supplement  the  technical  and  performance 
requirements  for  the  visual  system  a  set  of  four 
annexes  to  the  specification  were  developed. 

These  annexes  described  the  planned  use  of  the 
TCT  system.  These  annexes  were:  -•  -- 

Annex  A:  Representative  Task  List  and 
Generalized  Scene  Content 

Annex  B:  Concept  of  Training 

Annex  C:  Prototype  Visual  Data  Base 

Annex  D:  Maintenance  Concept 

The  annexes  were  developed  in  concert  with  TAC 
to  provide  a  thorough  orientation  and  a  ready 
reference  for  the  contractor  to  use  throughout 
the  program.  Annex  A  contains  a  comprehensive 
discussion  in  layman’s  language  of  each  task  to 
be  trained,  how  each  task  is  performed,  and  the 
visual  references  used  to  perform  the  tasks.  All 
tasks  and  discussions  were  oriented  toward  tacti¬ 
cal  (combat)  tasks  as  opposed  to  the  normal 
orientation  toward  initial  qualification  train¬ 
ing.  Most  of  the  TAC  training  requirements  are 
continuation  (recurring)  training  for  mission 
ready  tactical  fighter  pilots  which  consist 
primarily  of  tactical  tasks.  These  tactical 
tasks  have  a  significant  impact  on  the  visual 
system  requirements. 

To  document  his  particular  approach,  each 
contractor  was  required  to  prepare  a  Contractor 
Prepared  Prime  Item  Development  Specification 
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l^tOS;  which  w as  put  on  contract  and  updated  as 
is  cos  inn  became  bettor  ilefineo.  Annexes  A 
hroiKjh  D  wore  included  as  part  of  the  CPIDS. 


ihe  technical  requirements  imposed  upon  the 
ompetincj  contractors  fell  into  two  basic  cate- 
V0MGS:  (|)  Training  Task  Requirements  and  (2) 
System  Design  and  Performance  Requirements.  Th 


Task  Requirements  and  (2) 
o  nuance  Requirements.  The 


primary  drivers  were  the  training  tasks  and  the 
contractors  were  encouraged  to  be  innovative 
making  system  performance  tradeoffs  to  meet  the 


ami  ng  ta  s  k  requ  i  remen  t  s  . 


The  training  task  requirements  included  102 
tasks  which  may  be  grouped  into  six  basic  task 
rets.  These  basic  task  sets  are:  Air-to-Atr, 
Air-to-Surface,  Formation,  Low  Altitude  Tactical 
Navigation,  Aerial  Refueling,  and  Takeoff  and 
Landing.  Table  1  lists  the  basic  task  sets  and 
representative  tasks  within  each  set. 


TABLE  1  TRAINING  TASK  REQUIREMENTS  SUMMARY 
Ai  r- to -A i r 

Basic  Fighter  Maneuvers 

Basic  Counteroffensive  Maneuvers 

Gun  Tracking 

Missile  Employment 

Air  Combat  Tactics 

Low  Altitude  Intercepts/Engagement 

Visual  Missile  Trace  Use 

AAA/SAM/AI  Detect! on/Avo i dance 

Ai r- to-Surface 

Scorable  Range 
Box  Pattern 
Bomb,  Strafe,  Rockets 
Armed  Reconnai sance 
Forward  Air  Controller  Operations 
Tactical  Deliveries 
Pop-up 

Random  Attack 
Tactical  Bombing 
Moving  Target  Attack 
Guided  Weapons 

AAA/SAM/AI  Detection/Avoi dance 
Formation 

Takeoff,  Landing  (Lead/Wing) 

Close,  Route,  Trail 
Tacti cal 
Ai  r-to-Ai  r 
Ai r- to-Surface 

Low  Altitude  Tactical  Navigation 
Target  Acquisition 

Aerial  Refueling 

Tanker  Rendezvous 
Observation  Position 
Precontact  Position 
Contact  Position 

Takeoff  and  Landing 
Single  Ship 

Overhead  Traffic  Pattern 
Closed  Traffic  Pattern 
Ci rcl i ng  Approach 


Air  force  system  design  and  performance  re¬ 
quirements  were  primarily  written  in  terms  of 
performance  of  the  training  tasks  rather  than 
implementation  and  design  specification.  The 
only  implementation  directed  was  the  use  of  a 
CIG  system  as  an  image  source.  Special  precautions 
were  taken  to  avoid  precluding  potential  tech¬ 
nologies.  Table  2  lists  examples  of  the  basic 
requirements  grouped  into  categories.  The  re¬ 
quirements  were  later  specialized  by  each  of  the 
two  contractors  to  define  their  particular 
approach  to  solving  the  training  problem. 

After  contract  award,  unique  methods  were 
employed  to  insure  that  both  the  contractors  and 
the  users  fully  understood  the  significance  of 
the  system  requirements  throughout  the  program. 
Starting  during  the  second  month  of  the  contract. 
Contractor  Orientation  Visits  were  made  to  opera¬ 
tional  bases.  These  visits  included  opportunities 
to  look  at,  photograph  and  measure  the  aircraft 
and  weapons;  to  discuss  tasks  and  visual  cues 
with  pilots;  to  see  films  and  video  tapes  of  tasks 
being  performed;  and  to  ask  questions  of  the 
pilots  until  the  contractors  understood  the  tasks. 
No  questions  were  too  trivial  to  ask.  Orientation 
visits  were  held  with  A-10,  F-4,  F-15,  and  F- 16 
pilots  and  Tactical  Fighter  Weapon  Center  Pene¬ 
tration  Aids  Instructors  to  cover  all  aspects  of 
the  planned  use  of-the-T€T  system.  The  contractors 


TABLE  2  TCT  TECHNICAL  REQUIREMENTS/GOALS  SUMMARY 

System  Requirements 

Independent/ Interactive 
Correlated  In-Cockpit/Visual  Cues 
Object/Pattern/Light  Size,  Position 
AOI  (If  Proposed)  Position 
HUD  to  Visual  Alignment 
Geometric  Distortion 
Moving  Vehicles 
Visual  Effects 
Monochrome  (Color-Goal) 

AGM-65B  Maverick 
Channel  Misalignment 
Transport  Del  ay 
Safety 

Instructor  Operator  Station 
Avail  abil Ity/R&M  Goals 
Computational  System  Requirements 

CIG  Requirements 

Basic  Image  Content 
Expanded  in  Annex  C 
DMA  Terrain,  Cultural  Data 
Model  to  Meet  Training  Requirement 
Performance 
Accuracy 

Special  Effects  (Haze,  Clouds,  etc.) 

Display  Requirements 

Fi el d  of  Vi ew 
Collimatlon 

Head  Motion  Evelope  (Minimum) 


NOTE:  Contractors  were  encouraged  to  be 
innovative  and  to  make  tradeoffs  to 
meet  training  requirements. 


■“!0W(MI  extensive  learning  with  these  visits  and 
developed  a  strong  oper.tt. tonal  orienta t  ion.  This 
!'i,!  '•  w/y  positive  impact  on  both  system  analysis 
a n d  do  s  i  gn  ,  and  t  fie  mo  r a  1  e  o  f  t h o  pe  rs o n s  a  r.  s  i  gncd 
t.u  t^e  program.  A  true  mission  orientation 
■developed.  Hie  questions  hoc  time  more  incisive 
during  subsequent  visits  and  caused  the  pilots  to 
f  ea  1 1  _y  look  at  how  and  why  they  performed  each 
i  •  j  s  h .  P  i  1  o  t.  s  a  t  ten  d  e  d  e  a  c  h  d  e  s  i  gn  re  v  i  ew  t  o 
C o n 1 i n u o  t h e  opcratio n a  1  i n p uts.  ba t a  base  visits 
wrre  also  conducted  at  bases  to  be  modeled.  Data 
base  modelers  were  able  to  discuss  the  signifi¬ 
cance  of  feu  tu  res  with  pilots,  to  look  at  and 
:-hi«  to  graph  features,  and  to  collect  drawings  of 
avtield  layouts.  Reconnaisance  photography  of 
gunnery  ranges  and  low  altitude  tactical  noviga- 
t ' ! : 1 1  rou t e s  w as  also  p r o v  i  d e d  . 

ihe  orientation  arid  data  base  visits  made 
major  contributions  to  a  thorough  understanding 
of  the  requirements  and  to  t he  development  of  a 
sense  of  mission  on  the  part  of  contractor  per¬ 
sonnel.  Design  tradeoffs  were  then  made  after  an 
aria  lysis  of  the  effect  on  mission  performance 
instead  of  on  strictly  cost  or  technical  consid¬ 
er.!  t i ons . 

GENERAL  ELECTRICS  APPROACH 


General  Electric's  approach  to  the  TCT  visual 
system  was  based  upon  an  expanded  and  modernized 
Advanced  Simulator  for  Pilot  Training  (ASPT) 
display.  The  expansions  included  increasing  the 
field-o  f-view  (FOV)  to  provide  coverage  essen¬ 
tially  limited  only  by  aircraft  structure  and  the 
addition  of  a  High  Resolution  Area  (HRA)  inset 
into  the  lower  resolution  backgroud.  Figure  1 
"s  an  artist's  concept  of  the  TCT  IT  ST  as  designed 
by  GE  . 


The  visual  system  was  to  use  a  common  approach 
to  generate  and  display  all  imagery.  A  C.IG 
system  would  produce  eight  channels  of  video 
which  were  to  be  displayed  to  the  pilot  on  ten 
Ci'.  t  s  arid  viewed  through  a  dodecahedron  arrangement 
of  ten  pentagonal  Pancake  Window^*)  virtual- 
image  optical  units.  The  image  presented  to  the 
pilot  would  consist  of  a  helmet-slaved  high 
resolution  area  (HRA)  centered  within  a  large 
high  detail  background  area  of  interest  { AO  I )  and 
a  low  detail  background/horizon  filling  out  the 
remainder  of  the  display  as  shown  in  Figure  2. 

With  the  Maverick,  missile  selected,  one  channel 
of  video  would  be  dedicated  to  the  Maverick 
monitor  and  t he  remaining  seven  channels  allo¬ 
cated  to  the  visual  display.  The  two  or  three 
channels  winch  did  not  contain  CIG  imagery  would 
be  behind  the  pilot  and  would  have  a  horizon 
display  to  provide  a  peripheral  horizon  and 
illumination  in  the  cockpit. 

The  insetting  of  the  HRA  was  to  be  accom¬ 
plished  el ectroni cal  1 y  by  sequentially  scanning 
alternate  fields  of  the  f . ; 1  raster  and  a 
miniraster.  The  mini  raster  consisted  of  GOO 
scan  lines  and  600  pixels  per  scan  line.  The 
full  channel  raster  consisted  to  888  scan  lines 
with  888  pixels  per  scan  line.  The  fiel d-c f-view 
(FOV)  of  the  HRA  is  T5  degrees,  circular,  pro¬ 
viding  a  resolution  of  1.5  arc  minutes.  The 
average  resolution  of  the  full  channel  raster 
(for  both  high  detail  AOI  and  background)  is  6 
arc  minutes.  Both  the  HRA  and  high  detail  AOI 
display  the  same  high  detail  scene  content  and 
provide  most  of  the  visual  cues  to  the  pilot. 

The  low  detail  background  scene  contains  the 
terrain  outline,  horizon  and  representative 
alerting  cues  such  as  a  SAM  launch,  wingman  or  an 
air  interceptor.  The  high  detail  AOI  FOV  is  180- 


FIGURE  1  Artlstis  Concept  of  General 
Electric's  Approach  to  the  TCT  WST 


degrees,  horizontal,  and  80-degrees,  vertical. 

Hi  is  AO!  FO  V  is  based  upon  an  AfHRL  a  i  r-to-surface 
weapon  system  delivery  study  that  evaluated  pilot 
performance  for  various  rOVs .  The  HRA  remains 
centered  in  the  high  detail  AO!  and  both  are 
directed  about  the  total  FOV  of  the  display  by  a 
Helmet  Mounted  Sensor  (HMS).  Thus,  the  highest 
scene  content  will  be  displayed  to  the  pilot  in 
the  direction  his  head  is  pointed  with  the  high 
resolution  area  for  target  acquisition  centered 
within  the  area  of  the  highest  scene  content, 
ill  ending  is  provided  in  the  HRA  to  minimize  the 
effects  of  a  distracting  boundary  with  the  back¬ 
ground  imagery.  This  all  electronic  AOI  approach 
requires  no  electromechanical  servo  mechanisms. 
This  display  approach  has  no  constraints  on  the 
number  of  high  resolution  aircraft  or  ground 
targets  which  may  be  simultaneously  displayed  to 
the  pilot,  since  the  target  will  appear  at  high 
resolution  whenever  the  pilot  places  the  target-" 
in  the  HRA. 

While  the  helmet-slaved  AOI  approach  was 
adopted  to  avoid  false  "searchlight"  cues  which 
would  pinpoint  the  targets,  there  was  operational 
pilot  concern  that,  because  the  HRA  is  centered 
about  the  helmet  line  of  sight,  the  HRA  could 
not  be  directed  to  the  "6  o’clock"  position  and 
that  unnatural  visual  scan  patterns  would  be 
necessary  in  the  simulator  since  the  head  must 
be  moved  to  control  the  HRA  to  acquire  targets. 

It  should  be  noted  that  the  General  Electric 
visual  system  could  also  position  the  HRA  along 
the  pilot's  line  of  sight  (LOS)  by  using  both 
eye  and  head  position  measuring  devices. 

The  CRTs,  developed  by  Thomas  Electronics 
for  General  Electric,  are  36-inch  diameter  metal 
funnel  CRTs  with  a  dispenser  cathode  electron 
gun.  It  was  necessary  to  develop  the  metal 
funnel  CRT  technology  in  order  to  reduce  signifi¬ 


cant  schedule  risks  and  cost  associated  with  the 
use  of  glass  funnel  tubes  in  the  TCT  production 
phase  (as  well  as  for  safety  considerations  during 
manufacturing).  The  CRT  developed  for  the  TCT 
program  is  the  largest  metal  funnel  CRT  ever 
developed.  This  technology  is  presently  being 
used  to  fabricate  replacement  CRTs  for  both  ASPT 
and  Simulator  for  Air-to-Air  Combat.  Figure  3 
is  a  photograph  of  the  metal  funnel  CRT  in  process 
before  application  of  the  phosphor  screen. 

To  drive  the  CRTs,  the  display  electronics 
accept  the  video  signals  generated  by  the  CIG 
system  and  produce  signals  for  driving  each  CRT 
in  such  a  manner  that  the  proper  view  is  provided 
to  the  pilot.  The  geometric  distortion  require¬ 
ments  placed  upon  the  system  required  that  the 
CRT  sweep  signals  be  very  accurate.  The  display 
electronics  subsystem  designed  and  breadboarded 
by  GE  was  capable  of- providing  these  sweeps, 
including  the  compensations  for  CRT  to  image 
plane  mapping  and  Pancake  Window(R)  mapping 
functions.  The  magnitude  of  the  compensations, 
the  need  for  inset  raster  capability,  and  the 
accuracy  requi rements  led  to  an  approach  using 
linear  feedback  deflection  amplifiers  to  track 
precorrected  sweep  signals. 

Breadboard  CRT  electronics  were  fabricated 
and  tested  which  sequentially  scan  a  full  channel 
raster  field,  a  mini-raster  field,  a  full  channel 
raster  interlaced  field  and  mini-raster  inter¬ 
laced  field.  In  addition  to  the  dual  scanning 
format,  the  electronics  would  position  any  part 
or  all  of  the  inset  anywhere  within  the  channel 
FOV.  Figure  4  is  a  photograph  of  the  36-inch  CRT 
with  the  mini -raster  overwriting  the  full  channel 
raster  using  breadboard  CRT  electronics.  The 
mini-raster  is  scanned  in  a  square  format  as 
shown  in  the  figure.  The  blanking  for  the  round 
format  and  the  insetting  and  boundary  blending  is 


AREA  OF  INTEREST  TRACKS 
PILOT'S  LINE-OF-SIGHT 


LEFT  AZIMUTH  — RIGHT 
DEGREES 


FIGURE  2  The  Field-of-View  of  General  Electric's 
Visual  Display  and  Area-of-Interest 
(The  white  circle  represents  the  HRA) 
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generated  by  a  C i G  system.  Since  the  CIO  system 
generates  a  scene  with  a  "Tan  fi"  mapping  and  the 
Pancake  Window(R)  is  true  angle  mapping,  the  CRT 
electronics  must  compensate  the  raster  format  on 
the  CRT  to  display  an  und i s torted  image  to  the 
pilot.  This  compensation  also  requires  that  the 
circular'  ! ! R A  be  scanned  as  an  el  ipse  on  the  CRT 
as  the  HRA  moves  from  the  center  of  the  channel . 
tests  were  conducted  which  showed  that  the 
reg  i  s  fra  t  ion  error  on  the  HRA  to  t  fie  full  channel 
raster  was  less  than  0.10  degrees  within  the 
total  I OV ,  less  than  0.16  degrees  within  the 
instantaneous  fOV  and  was  much  less  near  the 

center  of  the  channel  .  Overal 1  ,  the  breadboard _ 

CR1  electronics  met  all  the  TCI  distortion  speci¬ 
fications  except  in  a  few  areas  at  the  edge  of 
: he  CRT.  In  addition,  the  breadboard  CRT 
electronics,  operating  with  a  36-inch  CRT,  met  or 
exceeded  the  resolution  and  MIT  requirements 
without  considering  the  effects  of  the  Pancake 
‘.•Jindow(R)  .  This  was  a  significant  advance  in  the 
s f a te-of- the-ar t  of  display  technology. 

i he  in  Line  Infinity  Optical  System  ( I L 1 0  S ) 
consisted  of  ten  f'arrand  Optical  Company  Pancake 
Windows'.'*)  and  a  dodecahedron  shaped  support, 
structure,  inch  optical  unit  has  a  24- inch  focal 
length;  and  is  pentagonal  !_v  shaped.  When  juxta¬ 
posed  in  the  dodecahedron ,  the  ten  windows  provide 
a  continuous  field  of  view  from  the  pilot's  eye 
reference  po in:.  'he  designed  and  manufactured 
1!.  [OS  is  similar  to  the  ASPT  system  with  two 
s i gn i f i cant  d i f f erenccs : 


a.  Although  the  dimensions  of  the 
individual  Pancake  Window^*)  and  dodecahedron 
are  the  same  as  the  ASPT  system,  t fie  orientation 
of  the  I L I  OS  relative  to  the  cockpit  is  different 
in  order  to  accommodate  three  additional  channels 
and,  therefore,  a  larger  TOV . 

b.  Improved  Pancake  Window^)  fabrica¬ 
tion  techniques  resulted  in  reduced  defects  and 
imperfections  in  the  optical  elements. 

The  overall  specified  performance  on  the 
General  Electric  visual  system  is  summarized  in 
Table  3.  Before  termi n action  of  the  contract,  Gf 
had  completed  and  tested  a  breadboard  of  the  CRT 
electronics  with  a  glass  funnel  CRT.  These 
breadboards  met  or  exceeded  all  TCT  specifi¬ 
cations.  A  36-inch  metal  funnel  CRT,  the 
largest  ever  built,  was  developed,  fabricated, 
and  tested.  When  tested  chis  CRT  met.  all  re¬ 
quirements  except  for  the  resolution  of  the  HRA  • 
raster.  Thi s  lower  resolution  was  attributed 
to  the  newly  designed  electron  gun  used  for  the 
test.  With  these  developments,  the  overall 
technical  risk  of  the  General  Electric  concept 
was  significantly  reduced.  However,  the  train¬ 
ing  risk  of  the  helmet-slaved  HRA  remains  and 
requires  further  evaluation  with  a  pilot  in  the 
loop.  Currently,  no  such  effort  has  been  funded 
to  resolve  this  question. 


FIGURE  3  Thirty-six  Inch  Metal  Funnel  CRT 
iri  Process  Before  Phosphor  Application 


FIGURE  4  The  High  Resolution  Mini-Raster  (Off  Center) 
Shown  Inset  Into  the  Background  Raster,  Visual  Display 

TABLE  3  GE  TCT  PERFORMANCE  PARAMETERS 


Total  Field  of  View: 

High  Detail  Area  of  Interest: 

High  Resolution  Area: 

Full  Channel  Resolution: 

High  Resolution  Area  (HRA)  Resolution: 
Brightness: 

Contrast  Ratio: 

Number  of  Edges: 

Number  of  Point  features: 

Number  of  Circular  Features: 

Video  Raster  Lines: 

Edge  Crossing/System  Line: 

Edge  Crossing/Channel  Paster  Line: 
Moving  Model  Types: 

f nvironrent : 

Scene  Enhancement.: 

Surface  lecturing: 

Visual  Data  Base: 

Gaming  Area: 


Essentially  same  as  aircraft 
120  degrees  Horizontal,  80  degrees  Vertical 
1 5  degrees  Circular 
6  Arc  Minutes,  Average 
1.5  Arc  Minutes 
6  rootlamberts ,  Highlight 
15:1  Minimum,  2.:1  Nominal 
4000 
4000 
1000 

888  Background,  600  Mini  Paster 
400  Background,  220  Miri  Raster 
256  Background,  220  Mini  Paster 
Airborne,  10  Types 
Ground,  50  Types 

full  Weather,  Time  of  Fay  Effects 
Articulated  Parts,  Curved  Surface  Shading 
Circular  features,  Curved  Surface 
Shading,  Point  features,  Gray  Shade 
Var ia t ions 

Both  Transformed  DMA  and  Hand  Models 
1380  N'M  x  1380  NM 


SINGER-LINK'S  APPROACH 


The  Singer-Link  approach  to  TCT  exploited 
the  fact  that  the  high-resolution  viewing  area 
of  the  eye  is  relatively  small.  This  high- 
resolution  area  is  the  fovea  of  the  eye,  which 
is  the  0"ly  area  where  small  details  may  be 
perceived . 

Surrounding  the  fovea  is  a  peripheral  area 
where  the  resolution  of  detail  is  low  but, 
because  of  the  way  human  vision  operates, 
there  is  a  high  sensitivity  to  movement.  The 
psychophysics  of  human  vision  creates  an  image 
in  the  "mind's  eye"  of  building  a  total  high- 
resolution  image  of  the  real-world  scene  from 
a  series  of  small  hi '-h-resol ution  "snapshots", 
each  of  which  is  surrounded  by  lower-resolution 
information.  If  this  situation  is  emulated  in 
the  visual  system,  the  FOV  requirement  for 
instantaneous  high  resolution  and  high  detail 
is  greatly  reduced.  Thus,  the  capacity  of  the 
image  generator  can  be  concentrated  where  it 
will  oe  used  ard  the  number  of  image  display 
channels  may  be  reduced. 

In  the  visual  system,  the  pilot's  line  of 
sight  (LOSj  is  monitored  and  a  high-resolution, 
nigh-detail  area;  surrounded  by  a  large  low- 
resolution,  low-detail  area;  is  displayed  along 
{‘■is  LCG.  .Then  the  eye's  LOS  changes,  this  is 


sensed  ar.d  the  high-resolution  area  is  moved 
accordingly,  matching  the  eye's  ability  to 
discern  high  detail  in  only  a  small  area  of 
the  total  scene  at  any  one  time.  The  net  result 
is  the  impression  that  there  is  high-resolution, 
high  detail  imagery  everywhere. 

The  TCT  eye-directed  AOI  visual  system 
provides  the  pilot  with  high  resolution  and 
high  detailed  imagery  anywhere  within  the  pilot's 
FOV.  This  is  accomplished  by  projecting  «  20 
degree  monochrome  image  on  a  35  ft  diameter 
dome  wherever  the  pilot  looks  out  the  cockpit. 
This  20  degree  fovea!  image  is  directed  alonq 
the  pilot's  LOS  using  servo  driven  fovea  1 
projectors.  This  foveal  image  is  inset  into  a 
lower  resolution  and  lower  detailed  monochrome 
peripheral  image  that  :s  projected  onto  the 
dome  through  fixed  wide  anqlp  peripheral  pro¬ 
jectors.  Figure  5  shows  the  layout  for  the 
projectors  coupled  to  an  A- 1 0  OFT.  It  requires 
four  foveal  and  four  peripheral  projectors  to 
provide  an  unobstructed  FOV  to  the  pilot. 

The  four  peripheral  projectors  are  located  at 
the  four  corners  of  the  simulator  ^45  degrees 
to  the  cockpit  axis  system)  and  are  optically 
merged  to  form  a  continuous  360  degree  image 
to  the  pilot.  However,  the  CIG  instantaneous 
field  of  view  is  limited  to  180  degrees  azimuth 
and  120  degrees  elevation  along  the  pilot's  LOS 
to  concentrate  the  scene  detail  capability  r 
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'he  Cin.  The  four  fovea  1  projectors  located  on 
Uie  cockpit  axis  system  (forward,  aft,  loft 
and  right)  have  overlap  on  the  dome  to  permit  the 
.^witching  of  fovea  1  projectors  without  loss  of 
imagery.  Therefore,  only  one  fovea  1  projector 
is  repaired  at  any  instand  to  provide  the  foveal 
image  along  the  pilot's  LOS.  The  imagery  for  both 
tiie  foveal  and  peripheral  projection  systems  is 
jenerated  by  a  single  five  channel  CIO  that 
supplies  874  active  lines  with  92;?  pixels  per 
line.  This  provides  better  than  1.4  arc  minute 
’’esolution  everywhere  within  the  total  simulated 
lire  raft  f iel d-o f-view.  In  addition,  two  very 
important  visual  cues,  head  motion  compensation 
:ind  realistic  G-dimming,  arc  inherent,  in  the 
eye-di rected  approach. 

in  real-image  displays,  there  is  an  inherent 
problem  of  image  placement  as  a  function  of  head 
notion.  In  previous  dome  displays  as  the  pilot 
noved  his  head,  the  simulated  imagery  would 
appear  to  be  located  on  the  projection  screen. 

The  eye-directed  system  solves  this  problem  by 
using  the  helmet  positional  information  to  move 
the  imagery  in  response  to  the  pilot  head  motion. 
This  results  in  correct  image  motion  so  that 
objects  located  in  infinity  move  as  if  they  were 
it  infinity  and  objects  at,  say,  5  feet  move  as 
if  they  were  at  5  feet.  This  is  accomplished  by 
providing  to  the  CIG  eyepoint  position  and 
orojector/screen/eyepoi nt  geometry  data.  The 
result  is  imagery  that  moves  as  it  should 
-dative  to  the  pilot  changing  viewpoint  not 
* n  1  y  with  respect  to  t fie  data  base  but  also  with 
respect  to  the  projection  geometry. 

The  visual  cue  associated  with  pul  lint  G's 
is  a  familiar  visual  effect  used  by  experienced 
o i lots  of  high  performance  aircraft.  As  the 
oilot  pulls  higher  G's  over  given  periods,  his 
r0Y  will  normally  start  to  collapse,  and  if  the 
G-level  is  sustained,  the  FOV  will  eventually 
collapse  to  zero.  This  collapse  could  be  fully 
simulated  in  the  TCT  visual  system.  The  initial 
collapse  down  to  iO  degrees  from  the  foveal  is 
handled  by  the  peripheral  hardware  with  the 
collapse  from  there  to  zero  being  handled  in 
the  foveal  merge  hardware.  In  an  air-to-air 
mission,  the  TCT  visual  system  will  allow  pilots 
to  "fly  the  tunnel"  when  engaging  another  aircraft 
at  short  range.  In  this  maneuver  the  pilot 
attempts  to  keep  the  target  within  the  fovea 
while  maintaining  sufficient  G's  to  provide  an 
almost  but  not  totally  collapsed  FOV. 

EYE  PSYCHOPHYSICAL  kEVIEW 

The  human  vision  system  provides  highly 
detailed  information  about  the  relative  positions, 
reflectances,  and  visible  emissions  of  very 
small  elements  in  the  visual  world.  It  also 
senses  changes  and  rates  of  change  in  scene 
element  positions,  and  supports  the  examination 
and  tracking  of  these  elements. 

Highly  detailed  vision  is  maximized  in  a 
relatively  small  area  in  the  center  of  the  eye, 
approximately  5  degrees  in  diameter,  known  as 
the  fovea.  The  eye  is  sensitive  to  light,  as 
far  out  as  90  degrees  from  the  fovea.  But,  in 
the  area  beyond  10  degrees,  the  functions  are 
primarily  to  alert  the  person  to  the  presence 
and  general  activity  of  the  scene  elements. 


Figure  6  depicts  the  resolution  failoff  of  the 
eye  as  a  function  of  angle  from  the  fovea. 

The  movement  of  the  eye  is  called  a  saccade. 
These  saccades  are  the  eye's  way  of  moving  new 
elements  into  the  fovea  for  detailed  examination. 
Saccadic  motions  are  typical-ef  voluntary  shifts 
of  attention  from  one  scene  element  to  another  and 
take  place  at  relatively  high  speeds  and  accel¬ 
erations.  The  speed  of  a  saccade  depends  on 
the  distance  to  be  scanned.  Experiments  indicate 
that  the  eye  tends  to  maintain  fixation  on  a 
scene  element  for  a  minimum  of  about  250  milli¬ 
seconds,  moving  to  new  scene  elements  with 
peak  velocities  of  700  degrees  per  second  with 
accelerations  of  about  50,000  degrees  per  second 
squared. 

During  these  high-speed  eye  movements,  scene 
information  is  passing  very  rapidly  across  the 
fovea  and  is  not  apparent  to  the  observer. 

This  visual  suppression  process  precedes  the 
start  of  a  saccade,  continues  on  through  the 
saccade  with  perception  not  fully  returning 
until  some  time  after  the  cessation  of  eye 
motion.  For  example,  one  is  unable  to  see  one's 
own  eye  movements  in  a  mirror  when  looking  from 
one  eye  to  the  other. 

The  previous  discussion  was  for  static  scenes 
only.  In  the  dynamic  scene  situation,  the  eye 
can  smoothly  maintain  points  of  interest  within 
the  fovea  for  objects  moving  up  to  about  100 
degrees  per  second.  During  these  tracking 
motions,  the  ;>cene  elements  of  interest  remain 
relatively  fixed  to  the  fovea  and  thus  there  is 
no  need  for  visual  suppression.  Even  so,  visual 
acuity  does  decrease  as  the  tracking  velocity 
increases.  At  high  velocities,  up  to  200 
degrees  per  second,  the  eye  uses  a  combination 
of  saccadic  motions  and  tracking  motions  to  main¬ 
tain  the  object  within  the  fovea. 


FIGURE  6  Relative  Visual  Acuity  as 
a  Function  of  Angle  from  Fovea 
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The  smooth  eye  motion  used  to  track  movinq 
objects  cannot  be  voluntarily  induced  without  the 
presence  of  moving  imagery.  It  is  difficult  to 
prevent  one's  eye  from  following  images  that 
a  large  portion  of  an  observer's  field  of 
view.  As  an  example,  one  can  observe  someone 
trying  to  move  his  eyes  smoothly  down  a  typed 
page.  It  cannot  be  done  without  the  aid  of  a 
pointer  smoothly  moving  down  that  page.  Also, 
if  the  observer  "focuses"  on  *  page  and  the 
page  is  moved  smoothly,  the  eyes  will  track  the 
page's  motion. 

DISPLAY  S V STEM 

The  design  of  the  TCT  visual  display  system 
was  based  on  these  two  characteristics: 

(1)  Visual  acuity  as  a  function  of 
angular  distance  from  the  fovea,  and 

(2)  Suppression  of  visual  perception 
during  eye  motion. 

The  first  cha racteristic  permits  the  concentration 
of  image  detail  and  resolution  in  the  region  where 
the  eye  is  looking.  Figure  6  suggests  that  high 
resolution  and  detail  in  the  peripheral  regions 
are  unnecessary.  The  suppression  of  visual 
perception  before,  during,  and  after  a  saccade. 
provides  the  key  to  the  feasibility  of  a  visual 
design  which  is  optimized  on  the  eve's  angular 
acuity  falloff.  Without  the  period  of  visual 
perception  loss  associated  with  a  saccade,  the 
bandwidth  requirements  to  position  the  foveal 
image  would  be  well  beyond  the  state-of-the-art. 

The  eye-director  AOI  display  approach  for 
TCT  was  developed  around  five  major  hardware 
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FI  CURE  7  Foveal  Projector  Diagram 


(1)  Foveal  projection  system 

(2)  Peripheral  projection  systems 

(3)  Helmet  mounted  oculometer  system 

(4)  Dome  display  screen 

(5)  CIG 

FOVEAL  PROJECTION  SYSTFM 

Figure  7  shows  the  components  of  a  single 
foveal  projector  from  video  input  to  the  dome 
screen. 

The  foveal  projection  lens  has  an  external 
pupil  and  is  approximately  telecentric.  The 
external  pupil  allows  the  servoed  mirror  to  be 
made  small  by  imaging  the  pupil  on  it.  The  first 
advantage  is  obvious:  the  servos  used  to  drive 
the  mirror  can  be  made  smaller  as  the  load  size 
is  made  smaller.  The  second  advantage  is  an 
increase  in  excursion  without  vignetting  as  com¬ 
pared  to  a  system  without  an  external  pupil. 

As  shown  in  Figure  7  there  are  five  servos 
in  the  foveal  projector;  two  control  LOS,  two 
control  size  and  focus,  and  one  controls  bright¬ 
ness.  The  servo  rates  and  accelerations  are  de¬ 
signed  to  be  consistent  with  the  requirements  of 
an  eye-directed  AOI.  Link  has  breadboarded  a 
complete  foveal  projector  that  fulfills  these 
requirements.  The  azimuth/elevation  servos  control 
the  LOS  of  the  foveal  image.  This  image  can  be 
pointed  anywhere  except  where  occulted  by  the 
projection  lens  or  azimuth/elevation  servos.  By 
the  nature  of  this  system,  the  image  must  be  de¬ 
fied  as  the  azimuth  servo  rotates;  this  function 
is  accomplished  in  the  image  generator.  The  zoom/ 
focus  servos  control  the  size  and  focus  of  the 
projected  foveal  image.  Due  to  the  geometry  of 
the  display  system,  the  throw  distance  and  apparent 
AOI  FOV  size  to  the  pilot  will  vary  as  a  function 
of  his  LOS.  In  order  to  maintain  a  constant 
AOI  FOV  size  to  the  pilot,  the  projected  FOV  must 
be  varied  as  a  function  of  the  pilot  LOS.  In 
addition,  the  image  must  be  refocused  as  the 
projection  throw  distance  varies.  Both  these 
functions  are  controlled  by  zoom  and  focus.  The 
attenuator  servo  controls  the  brightness  of  the 
projected  image.  Becajise  of  the  geometry,  the 
brightness  of  the  imagery  before  reaching  the 
screen  must  be  varied  as  a  function  of  the  pilot 
LOS  in  order  to  maintain  a  constant  apparent 
brightness  to  the  pilot.  The  main  contributing 
factors  to  the  variance  are  (1)  screen  gain,  (2) 
angle  of  incidence  and  reflectance  with  reject 
to  the  pilot ,  and  (3)  projected  FOV.  The  bright¬ 
ness  of  the  output  image  is  controlled  by 
"otating  a  variable-density  disk  located  within 
the  optical  path. 

The  light  source  for  the  foveal  projector 
will  be  provided  by  a  GE  light  valve  projector 
(PJ7150).  This  light  valve  provides  a  light 
output  of  1,000  lumens  with  a  minimum  resolution 
of  800  by  750  TV  lines  per  picture  height 
resolution.  The  light  valve  will  be  modified  to 
a  1:1  format  rather  than  the  standard  4:3  format, 
to  reduce  the  losses  in  foveal  image  circular 
FOV.  Tie  modifications  will  result  in  a  usable 
light  output  of  750  lumens  and  a  resolution  of 
800  TV  lines  horizontal  and  750  TV  lines  vertical. 
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The  brightness  of  the  outside  3  degrees  of 
the  fovea!  image  is  reduced  to  blend/merge  the 
fovea!  image  into  the  peripheral  imagery.  This 
blending  is  accomplished  by  electronically 
feathering  that  area  of  the  fovea!  system  as  a 
function  of  angular  subtense  to  the  pilot  eye- 
point.  The  reverse  is  done  in  the  peripheral 
merge  electronics. 

PERIPHERAL  PROJECTION  SYSTEM 

Figure  8  shows  one  of  the  four  peripheral 
projection  systems  from  video  input  to  the  dome 
screen . 

The  system  utilized  a  wide  angle  lens  in  each 
peripheral  projector  to  provide  information  in 
the  peripheral  FOV.  The  output  is  a  210  degree 
(relative  to  the  lens)  circular  field  projected 
onto  the  dome  surface.  Within  the  optical  chain 
of  the  peripheral  projector,  a  variable-density 
(spatial  only)  filter  will  be  used  to  provide  two 
functions.  The  first  is  the  flattening  of  the 
field  brightness  as  seen  by  the  pilot.  This 
comparison  is  required  due  to  lens  falloff, 
light  valve  falloff,  screen  gain,  and  varying 
bend  angles.  The  second  use  is  the  feathering 
of  the  peripheral  image  at  the  boundary  between 
two  peripheral  projectors.  The  roll  assembly 
is  used  to  orient  the  4:3  aspect  ratio  of  the 
light  valve  within  the  circular  optical  FOV. 

The  light  source  for  the  peripheral  projector 
will  be  provided  by  a  GE  light  valve  projector 
(PJ7155).  The  light  valve  provides  a  light  out¬ 
put  of  2,000  lumens  with  a  minimum  resolution  of 
800  by  750  FV  lines  per  picture  height  resolution 

As  mentioned  in  the  foveal  projector  descrip¬ 
tion,  the  outside  3  degrees  of  the  foveal  image 
is  used  to  b1  cn!/:rerge  the  foveal  image  into  the 
peripheral  imagery.  The  blending  is  accomplished 
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by  electronically  feathering  that  area  of  the 
peripheral  as  a  function  of  angular  subtense  to 
the  pilot  LOS.  In  addition,  the  area  inside  the 
blend/merge  region  is  electronically  blanked 
within  the  peripheral  to  provide  a  blank  hole 
for  insertion  of  the  foveal  image. 

The  projection/screen/eyepoint  geometry 
along  with  the  wide-angle  lens,  would  provide  a 
distorted  image  to  the  pilot  if  left  uncorrected. 
TCT  utilizes  two  methods  of  correction  in  order  to 
reduce  the  complexity  of  the  system.  The  first 
approach  is  software  mapping  within  the  image 
generator.  This  removes  the  basic  distortion 
introduced  due  to  the  projector/eyepoint  geometry. 
The  second  method  of  correction  is  provided  by 
a  scan  converter.  The  scan  converter  compensates 
for  the  remaining  distortion  caused  by  the  wide- 
angle  optics  and  spherical  screen. 

HELMET  MOUNTED  OCULOMETER  SVSTEM 

The  measurement  of  the  pilot's  eye  LOS 
utilizes  a  Helmet-Mounted  Oculometer  System  (HM0S) 
developed  by  Honeywell  Avionics  Division  under  a 
sub-contract  to  Singer.  The  eye  LOS  relative  to 
the  cockpit  is  continuously  measured  by  the  HM0S 
in  two  stages:  (1)  A  standard  Honeywell  Helmet- 
Mounted  Sight  (HMS)  measures  helmet  position  and 
LOS  relative  to  the  cockpit,  and  (2)  a  recently 
developed  Honeywell  Helmet-Mounted  Oculometer 
(HM0)  measures  eye  LOS  relative  to  the  helmet. 

The  two  LOS  are  then  summed  to  obtain  the  eye  LOS 
with  respect  to  the  cockpit.  The  resultant  eye 
LOS  and  helmet  position  is  then  provided  to  the 
display  computer  to  control  the  position  of  the 
foveal  image.  This  same  data  is  provided  to  the 
CIG  for  the  generation  of  the  foveal  and  periph¬ 
eral  image.  Figure  9  depicts  the  components 
mounted  on  the  helmet. 

The  HMS  subsystem  utilizes  a  magnetic  field 
established  by  a  transmitter  located  behind  and 
above  the  pilot's  head.  The  magnetic  field 
components  are  detected  by  a  receiver  mounted  in 
the  helmet  as  shown  in  Figure  9.  The  amplitudes 
of  the  field  components  are  then  transformed  into 
helmet  position  and  LOS. 


FIGURE  8  Peripheral  Projector  Diagram 


FIGURE  9  Helmet  System  Layout 
Mounted  Oculometer 
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The  HMO  main  component  is  a  charge  coupled 
device  (CCD)  camera  that  views  the  pilot's  eye 
from  a  mounted  position  on  the  helmet,  Figure  10. 
Ihe  pilot's  eye  is  illuminated  by  a  low  intensity 
IN  lamp  that  shares  the  same  optical* path  with 
the  camera  through  a  beam  splitter.  The  CCD  picks 
up  the  illuminator's  reflection  off  of  the  pilot's 
cornea  along  with  the  pilot's  pupil  image.  Using 
this  information,  the  HMO  system  determines  the 
pilot's  eye  LOS  with  respect  to  the  helmet.  The 
HMO  computational  system  then  combines  the  HMS 
helmet  LOS  with  the  HMO  eye  LOS  into  a  combined 
pilot's  eye  LOS  with  respect  to  the  cockpit. 

DOME  DISPLAY  SCREEN 

The  display  screen  is  constructed  of  three 
basic  fiberglass  panels  and  rib  structures.  The 
panels  below  the  equator  are  22.5  degrees  x  46.7 
degrees;  above  the  equator  the  panels  are  22.5 
degrees  x  62  degrees;  and  the  cap  panels  are  22.5 
degrees  x  28  degrees.  The  screen  finish,  a 
patented  Link  process,  provides  a  variable  gain 
function  in  order  to  provide  compensation  for 
brightness  fall  off  due  to  the  projector  and 
observer  locations.  The  screen  gain  is  four  above 
'55  degrees  elevation  and  two  below  +10  degrees 
elevation.  The  gain  varies  linearly  from  two  to 
four  with  elevation  angles  between  +10  degrees 
and  +55  degrees.  The  dome  construction  process 
entails  assembling  the  rib  structure  then  laying 
in  the  panels.  The  panels  are  riveted  and 
epoxied  in  place.  The  seams  between  panels  are 
filled  to  the  dome  radius  before  the  multi  process 
screen  coating  is  applied. 

COMPUTER  IMAGE  GENERATOR 

A  single  five  channel  CIG  system  drives  the 
four  peripheral  projectors  and  four  fovea! 
projectors. 

Four  image  channels  are  continuously  projected 
on  the  dome  screen  to  provide  the  peripheral 
image.  The  scene  content  is  allocated  between 
channels  based  upon  the  eye  LOS.  The  fifth 
image  generator  channel  is  switched  between  the 
four  fovea!  projectors  as  a  function  of  the  pilot 
LOS  outside  the  cockpit.  In  addition,  the 
coveal  image  channel  is  used  for  the  Maverick 
cockpit  display  monitor  when  the  pilot's  LOS  is 
directed  toward  the  monitor. 

The  foveal  image  channel  can  process  3,000 
potentially  visible  edges  at  a  rate  of  sixty 
times  a  second;  the  four  peripheral  image 
channels  can  process  a  total  of  6,000  potentially 
visible  edges  at  a  rate  of  thirty  times  a  second 
in  an  instantaneous  F0V  of  180  degrees  x  120 
degrees  about  the  pilot's  foveal  LOS. 

STUDIES 

During  the  TCT  contract,  Singer  conducted 
several  studies  to  define  critical  system 
parameters.  The  test  arrangement  consisted  of 
viewing  a  rear  projection  screen  with  an  eye 
directed,  high  resolution  A0I  in  a  low  resolution 
background.  The  image  was  generated  by  two 
television  cameras  viewing  test  imagery.  One 
camera  operated  at  full  resolution  while  the 
second  camera  was  operated  at  approximately 
one- fifth  resolution.  A  bench  mounted  oculometer 
detected  eye  position  and  directed  the  position 
c  r  a  variable  size  and  blend/merge  of  the  AO  I 
(from  the  first  television  camera)  inset  into 


the  background  (from  the  second  camera).  A  GE 
light  valve  then  projected  the  image  on  the  screen 
for  viewing. 

The  AO  I  FOV  was  variable  from  10  to  24  degrees. 
The  blend  region,  within  the  A0I,  was  adjustable 
for  zero  blend  and  variable  from  1.4  to  6.7 
degrees . 

In  these  studies,  a  blend  region  of  3  to  4 
degrees  within  the  A0I  provided  the  best  viewing. 
With  this  blend  region,  the  effects  of  image  mis¬ 
match,  brightness  and  resolution  differences  were 
minimized.  A  high  resolution  area  of  12  degrees, 
exclusive  of  blends,  provided  the  best  viewing. 

This  value  maybe  decreased  as  the  eye  tracking 
accuracy  improves.  Thus,  the  final  A01  design 
was  a  14  degree  FOV  of  high  resolution  imagery 
surrounded  by  a  blending  ring  of  3  degrees  for 
a  total  A01  FOV  of  20  degrees. 

An  additional  benefit  of  these  studies  was  the 
unstructured  subjective  evaluation  of  the  eye 
directed  A0I  concept.  As  the  observer  moved  his 
eyes  throughout  the  scene,  it  did  appear  to  be 
completely  high  resolution.  However,  when  the 
A01  was  not  eye  directed,  the  difference  between 
A0 r  and  background  resolution  was  easily  seen. 

This  provided  additional  confidence  that  this 
concept  would  be  successful . 

Prior  to  the  termination  of  the  contract, 

Singer  completed  the  fabrication  and  test  of  the 
HM0S ,  Figure  10.  Two  units  were  built  and  both 
met  or  exceeded  the  TCT  requirements.  In  addition, 
a  breadboard  of  the  foveal  projector  was  fabricated 
but  not  tested,  Figures  11  and  12.  This  breadboard 
was  tested  on  a  subsequent  contract,  Project  EDIT, 
described  below.  These  t^sts  indicated  that  all 
TCT  requirements  were  met,  except  for  the  peak 
velocity  of  the  azimuth  servo  requirement.  Cor¬ 
rections  to  this  problem  have  been  designed  and 
will  be  incorporated  in  the  future.  The  perfor¬ 
mance  of  the  Singer  eye-directed  A0 1  approach  is 
summarized  in  Table  4. 


FIGURE  10  Helmet-Mounted  Oculometer  System 


PROJECT  EDIT 


FIGURE  11  Fovea!  Image  Projector 


Project  EDIT  (Eye-Slaved  Display  Integration 
and  Test)  is  a  cooperative  Air  Force  and  Navy 
continuation  of  the  Singer  eye-directed  AOI 
development  started  on  TCT,  The  Air  Force  is 
providing  residual  TCT  contract  equipment  and 
funding.  The  Navy  is  providing  funds,  equipment, 
facilities,  and  contract  administration.  Project 
EDIT  is  a  multiphased  effort  which  will  lead  to 
pilot  evaluation  of  the  TCT  eye-directed  AOI 
technology  on  the  Visual  Technology  Research 
Simulator  (VTRS)  located  at  the  Naval  Training 
Equipment  Center. 

Phase  I  has  been  recently  completed  where 
the  HMOS  and  servos  were  integrated  with  the 
central  computer  and  dynamically  tested;  and 
the  video  electronics  checked  out.  and  dynamically 
tested.  Phase  II  started  in  August  1982.  During 
this  phase,  all  the  subsystems  will  be  integrated 
and  tested  end-to-end.  Tin's  will  include  using 
an  artificial  eye  to  provide  known  inputs  to  the 
HMOS  and  measuring  the  response  of  the  projector 
servos.  Preliminary  plans  are  that  Phase  III 
will  integrate  the  breadboard  eye-directed  AOI 
equipment  on  the  VTRS  for  pilot  evaluation.  The 
exact  plan  and  schedule  for  Phase  III  will  depend 
upon  available  funding  and  success  during  the 
various  stages  of  integration. 

The  Air  Force  goal  for  Project  EDIT  is  to 
determine  the  feasibility  of  the  eye-directed 
AOI  concept  for  tactical  combat  training.  It' 
successful,  data  from  Project  EDIT  will  be  used 
to  establish  system  performance  specifications 
for  future  procurements.  The  eye-directed  AOI 
approach  has  the  potential  of  meeting  most  of 
the  tactical  training  requirements. 

CONCLUSIONS 

Although  the  TCT  contracts  were  terminated, 
several  developments  were  successfully  completed, 
such  as  the  36-inch  metal  funnel  CRT  and  the 
Helmet  Mounted  Oculometer  System.  These  equip¬ 
ments  are  currently  being  used  in  laboratories 


TABLE  4  SINGER  TCT  PERFORMANCE  PARAMETERS 


Field  of  View: 

Resol uti on : 

Contrast  Ratio: 

Bri ghtness  : 

Foveal  FOV : 
Peripheral  FOV: 

Number  of  Edges: 

Texture : 

Moving  Model  Types: 
Envi ronment : 

Gaming  Area: 

Scene  Enhancement: 
Visual  Data  Base: 
Special  Effects: 


Total  aircraft  FOV  including  head  motion 

1.4  arc  minutes 
14:1 

1.5  foot  Lamberts 

20  degrees  diameter  with  3  degrees  merge 
360  degrees  Ht+90  degrees,  -60  degrees  V 
except  where  occulted  by  cockpit 
1  foveal  channel,  3,000  edges  @  60  Hz 
4  peripheral  channel s,  6 ,000  edges  @  30  Hz 
Surface  (16  unique  patterns) 

Airborne,  10;  Ground,  50 
Full  weather,  time  of  day  effects 
1,000  x  1,000  nautical  miles 
Articulated  parts,  light  point  features 
Transformed  DMA,  Hand  modelled  , 

Head  Motion  Compensation 
G- Dimming 
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to  per  form  much  needed  rose arch.  In  addition, 
the  feasibility  of  other  developments  were 
demons tra ted  such  as  the  inset  mini-raster  on  a 
CRI  and  the  foveal  projector.  It  is  hoped  that 
work  will  continue  to  complete  and  evaluate  these 
developments. 

i On  the  TCI  program,  there  was  a  continuing 
evaluation  of  the  Technical  and  Training  Risks 
of  fielding  a  successful  WST.  It  is  our  belief 
that  the  work  accomplished  on  these  contracts 
and  described  in  this  paper  have  significantly 
reduced  the  Technical  Risk  of  these  two  approaches. 
However,  continued  effort  is  needed  to  reduce 
tnese  Technical  Risks  for  a  production  procure¬ 
ment.  Additionally,  the  Training  Risks  have  not 
been  significantly  reduced.  A  simulator  environ¬ 
ment  for  flying  combat  tasks  is  needed  to  permit 
task-loading  the  pilot  to  predict  the  training 
performance  of  the  visual  system.  Continued 
efforts,  such  as  Project  EDIT  and  other  similar 
work,  are  needed  to  reduce  the  Training  Risks. 


FIGURE  12  Closeup  of  Azimuth/Elevation 
Servo  Assembly 


Another  contribution  of  the  TCT  program, 
although  not  strictly  technical,  is  demonstrating 
the  importance  of  thoroughly  communicating  to 
the  design  engineers  the  pilot’s  tasks,  visual 
references  and  how  such  visual  references  are 
used.  The  Air  Force,  General  Electric  and 
Singer-Link  agree  that  the  orientation  trips 
and  the  continued  dialogue  between  designers  and 
pilots  during  the  TCT  program  was  extremely 
useful  in  designing  the  TCT  system  and  determining 
design  and  performance  tradeoffs.  It  is  hoped 
that  these  effective  communication  techniques 
will  be  used  on  future  programs. 

References : 

Hutton,  D.  P.,  Burke,  D.  K. ,  Englehart,  J.  D. , 
Wilson,  J.  M.  ,  Romaglia,  F.  M.,  and  Schneider, 

A.  j.,  " Ai r-to-Ground  Visual  Simulator 
Demonstration  Final  Report"  (2  volumes). 
Aeronautical  Systems  Division,  Simulator  System 
Program  Office,  Wri ght-Patterson  Air  Force  Base, 
Ohio:  October  1976 

Monroe,  E.  G.,  and  Rid.eson,  W.  E.,  "CIG  Edge 
Conservation  Evaluation  and  Application  to  Visual 
Flight  Simulation"  Proceedings,  Tenth  NTEC/ 
Industry  Conference,  November  1977 

ABOUT  THE  AUTHORS 

Mr  James  D.  Basinger  was  the  Chief  Engineer 
on  the  TCT  pi'ogram  for  the  Aeronautical  Systems 
Division,  Deputy  for  Simulators.  His  current 
responsibilities  are  Senior  Systems  Engineer  on 
the  C- 130  Visual  System  and  Research  and  Develop¬ 
ment  Planning  Engineer.  He  has  been  associated 
with  Air  Force  visual  simulation  research  and 
development  programs  since  receiving  his  B.S.E.E. 
degree  from  Ohio  Northern  University. 

Mr  John  M.  Wilson  was  the  Deputy  Chief 
Engineer  on  the  TCT  program  for  the  Aeronautical 
Systems  Division,  Deputy  for  Simulators.  He  is 
currently  the  Senior  Systems  Engineer  for  the 
EF-111A  OFT.  He  has  been  associated  with  visual 
simulation  research  and  development  programs  with 
industry  and  the  Air  Torce  for  twelve  years. 

He  has  extensive  Air  Force/Air  National  Guard 
experience  as  a  tactical  fighter  pilot.  Mr  Wilson 
has  a  BS  Aero  Engineering  Degree  from  the 
University  of  Minnesota  and  a  MS  Aero  Astro 
Engineering  Degree  from  the  Ohio  State  University. 

Mr  Robert  A.  Fisher  is  a  Visual  Systems 
Program  Engi  neer  with  Singer  Link,  Flight. 
Simulation  Division.  He  is  currently  Principal 
Investigator  for  the  Wide  Angle  Tactical  Visual 
Program  and  Program  Engineer  on  the  Eye-Slaved 
Integration  and  Test  Program.  He  has  over 
twenty-two  years  experience  in  display  system 
technology.  Mr  Fisher  has  a  B.S.E.E.  degree 
from  Syracuse  University. 


230 


2 


a  *  ?(j  Aps  c: 

t\*:  > -'tf  vs--- 


f*  n.pY 


AD  P00018 


DATA  BASE  GENERATION  SYSTEM  FOR  COMPUTER  GENERATED  IMAGES 
AND  DIGITAL  RADAR  LANDMASS  SIMULATION  SYSTEMS. 

\  By  Lt.  Col.  Manfred  Haas. 

\  Dlether  Elflein  and  Peter  Gueldenpfennig 

ABSTRACT 

’“The  paper  deals  with  a  semi  automatic,  interactive  system  to  generate  data  bases  from  Digital  Landmass 
System  (DLMS)  Data,  for  Computer  Generatecidmage  Visual  Systems  (CGIVS)  and  for  Digital  Radar  Landmass 
Simulation  (DRLMS)  Systems. 

Terrain  information  and  certain  culture  features  can  be  gained  from  DLMS  data  automatically  for  CGIVS  and 
DLRMS  data  bases.  Additional  information  is  prepared  by  interactive  methods,  including  the  use  of  model  library 
for  CGI  data  bases  developed  by  batch  procedu'es.  Data  bases  can  also  be  developed  solely  by  batch  pro¬ 
cedures. 


INTRODUCTION 

In  1975  the  German  An  force  and  Navy  decided  to  use  a  Com¬ 
puter  Generated  Image  Visual  System  (CGIVS)  and  a  Digits’ 
Radar  Landmass  Simulation  (DRLMS)  System  for  the  TOR 
NADO  Operational  Flignt  Training  and  Tactics  Simulator.  The 
development  of  the  prototype  CGIVS  demonstrated  that  in 
order  to  fullfill  the  operational  requirements  for  this  type  of 
simulator,  a  substantial  mcrjase  in  scone  content  would  be 
necessary.  Therefore,  the  CGIVS  production  units  have  a 
much  higher  data  processing  capability  than  the  prototype. 
Table  1  shows  a  comparison  of  the  capabilities  of  the  pro¬ 
totype  CGIVS  and  production  units. 

prototype  or,  l  vs  PROP  JCHONS  CGIVS 

EDGES  SCENE  <XX>0  8000 

POINT  IIGHTS-SCENE  KXIO  4LXX> 

FACESSCENE  SOO  <000 

REAL  TIME  DATA  RASE 

CAPACITY  10.000  40 000'  Plux  0>f<A(rir 

Rrioadmc 

TV  LINES  VS  «,'•> 

RASTER  FLEMENTS.TV  UNE  Si?  1000 

TEXTURING  NO  YES 

CURL EP  SURFACE  SHADING  NO  YES 

T.jMi*  1 

Cortipari  of  Proto!  %  p<>  and  Production  CGIVS 
fo'-  tht*  1  mining  Simulator  TORNADO 


The  increased  scene  content  tor  the  new  CGiVS  had 
substantia!  impacl  on  data  base  generation  For  the  prototype 
system,  the  CGIVS  data  bases  had  been  generated  manually 
by  batch  operation  The  areas  and  models  had  been  derived 
from  geodatic  charts,  areal  photographs,  blue  prints,  and  nor¬ 
mal  photographs  The  scene  content  of  this  material  was 
reduced  and  manually  transformed  into  graphic  vectors  The 
coordinates  of  the  vertices  of  the  edges  were  defined  on  pun 
ched  cards  Only  after  the  coordinates  of  the  vertices  had  been 
defined  on  the  punched  cards  automatic  data  processing 
could  be  used  for  operations  such  as  reading  in.  testing,  seal 
mg  and  computing  o?  iace  normals  and  separation  planes 

The  high  information  density  of  the  data  bases  to r  the  new 
CGIVS  led  to  tne  necessity  for  use  of  automatic  and  interac 
tive  procedures  for  the  development  of  data  bases  For  this 
new  CGIVS.  Messerschmitt  Bolkow  Blohm  (MBB),  designed 
the  Data  Base  Generation  System  (0BGS)  This  DBGS  will  not 
only  be  used  tor  the  design  of  data  bases  for  CGIVS.  it  will  also 
perform  generation  and  modification  of  data  bases  for  the 
DRLMS  system  it  i$  worth  mentioning  that  alt  DRLMS  for  the 
German  and  Italian  TORNADO  simulators  have  an  update  con 
sole  with  which  small  modifications  to  the  data  bases  can  be 
performed  One  system  out  of  the  six  ordered  by  the  German 
Government  and  one  of  tn«  Italian  Airforce  systems  has  addi¬ 
tional  computer  peripherals  used  to  transfo  m  source  material 


into  online  data  bases.  The  source  material  for  this  transfor¬ 
mation  program  is  cartographic  information  in  digital  form  in 
accordance  with  the  product  specification  for  Digital  Land- 
mass  System  (DLMS)  Data  Base/ICD/100  1  Edition.  July  1977. 


DEVELOPMENT  OF  VISUAL  DIGITAL  DATA  BASES 
Structure  of  The  Visual  Data  Bases 

The  data  bare  for  the  CGIVS  represents  the  mathematical 
description  of  the  stylized  world.  This  description  is  in  ceded 
form  on  a  storage  medium,  so  that  the  CGIVS  can  read  it  out  in 
an  online  process  and  can  perform  further  data  processing 
tasks 

The  geometry  of  the  •'world"  is  defined  by  points,  lines  and 
faces.  A  point  is  defined  by  storage  of  its  X  Y-.  and 
Z-coordmates  Two  points  define  a  line  By  the  use  of  several 
lines  a  closed  polygon  can  be  developed  which  represents  a 
face.  The  CGIVS  shows  points  and  faces  which  are  described 
not  or'y  by  their  geometrical  position,  but  by  other  attributes 
like  color,  texturing,  or  curved  surface  shading. 


F  i •.!»•*»  t.  ■*»•*-«•  ,«u  im.t!  !«•** 


■o  Thf.  morphology  of  the  terrain  is  approximated  by  triangles 

ed  to  0th?  1'  °l!tl,neS  and  position  of  ‘he  triangles  are  match-’ 
td  to  the  terrain  in  an  optimum  manner.  The  number  of 

triang  es,  and  therefore  the  information  density  of  the  data 
»h«  f  uncn?,n  of ‘he  roughness  of  the  terrain.  This  means 
that  th„  information  density  of  a  planar  landscape  Is  relatively 
small,  and  for  mountainous  terrain,  it  is  very  large. 


It  is  possible  to  divide  fhese  triangles  into  more  faces,  in 
,  er  t0  show  changes  of  ground  vegetation  and  culture 
features  by  the  means  of  different  attributes. 


Three-dimensional  objects  can  be  built  out  of  these  faces. 
These  objects,  in  turn,  form  more  complex  models.  Objects  are 
always  convex,  while  models  can  also  have  a  concave 
character.  Objects  and  models  can  represent  houses,  towers 
bridges  etc.,  which  are  positioned  in  the  terrain. 

In  principle  the  DDGS  enables  three  methods  of  data  base 
generation:  Automatic  transformation,  interaction  and  batch 
(See  figure  2). 
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Figure  2.  Data  Base  Generation  System 
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Figure  2,  Hardware  Conf  igurat  ion 


Hardware  Configuration  (See  figure  3) 

Iht>  Diitn  Ruso  Generation  System  is  a  stand-alone  corn- 
putof  complex  with  computer  peripherals.  It  can  be  operated 
independently  from  the  simulator  and  consists  ot  three  main 
groups.  Computer  System.  Display  System,  and  Interaction 
system  Only  commercial  hardware  is  used.  The  system  has 
the  following  configuration: 


a.  One  Perkin  timer  8/32  Minicomputer  with  1  MB  Func¬ 
tional  Storage,  Writable  Control  Store  and  Floating  Point 
Unit. 

b  One  type  AP  i?Ob  Array  Processor  developed  by  Floating 
Point  for  parallel  performance  of  time  consuming  pro¬ 
cedures. 

e.  Two  Perkin  Elmer  Type  550  CRT  terminals  which  enable 
simultaneous  communication  with  tire  compute 

d.  One  card  reader  (1000  CPU)  and  one  line  printer  (300  LPM), 
developed  by  Perkin  Elmer  as  additional  input/output 
peripherals. 

e  Two  9  track.  1600  bpi  mag-tape  units  for  the  input  of  large 
amounts  of  data. 

f.  Three  magnetic  300  MB  disc  drives  for  the  storage  of 
completed  data  bases. 

g  Two  interactive  stations,  each  consisting  of  one 
Tektronix  4014- 1  graphics  terminal,  and  one  Tektronix  4954 
digitizing  tablet  for  parallel  graphical  data  processing.  The 
picture  content  of  one  graphics  terminal  can  be 
documented  using  one  Tektronix  4631  hard-copy  unit. 

This  hardware  configuration  enables  the  transformation  of 
DMA  data,  interactive  data  base  generation  and  modification, 
as  well  as  batch  processing  for  the  generation  and  modifica¬ 
tion  of  data  bases  for  CGIVS  and  for  DRLMS. 

VISUAL  SIMULATION  SYSTEM/DATA  BASE  GENERATION 

Data  Base  Sources 

The  source  material  for  the  Data  Base  Generation  System  is 
digital  iandmass  data.  To  satisfy  the  tremendous  requirment 
for  such  data,  the  military  geographical  agencies  of  the  NATO- 
countries  have  defined  one  standardized  format  in  which  the 
information  on  geographical  charts  is  digitally  stored. 

There  are  two  types  of  DLMS  data:  terrain  data  and  culture 
data.  This  information  for  the  same  area  is  stored  on  two  dif¬ 
ferent  magnetic  tapes.  The  terrain  data  represent  altitude  data 
at  the  cross-points  of  a  grid  system.  The  grid  systems  are 
divided  into  five  different  zones  according  to  latitude,  so  that 
each  square  has  an  edge  length  of  100  meters  for  the  coarser 
level  of  detail  (level  t).  and  30  meters  for  the  finer  level  of  detail 
(level  2). 

The  culture  data  describes  the  natural  terrain  structure  and 
the  artificial  cultural  features  of  the  terrain.  There  are  points, 
lines,  and  faces  defined,  which  are  numbered  by  an  analysis 
code.  Information  concerning  the  geographical  position  and 
dimension,  as  well  as  the  material  category  and  the  identifica¬ 
tion  code,  is  attached.  Thirteen  different  material  categories 
represent,  for  example,  water,  earth,  rock,  metal,  etc.  By  the 
use  of  the  identification  code,  characteristics  can  be  differen¬ 
tiated  in  vpry  great  detail.  For  example,  different  shapes  of 
roofs  or  bridges  can  be  discerned. 

Terrain  Transformation 

An  algorithm  is  used  in  the  Data  Base  Generation  System 
which  enables  the  triangulation  of  DLMS  terrain  data  in  a  way 
that  the  advantage  of  variable  information  density  can  be  us¬ 
ed.  The  logic  of  this  algorithm  is  based  primarily  on  "trial"  and 
"error".  The  trials  are  performed  for  a  certain  number  of 
DRLMS  grid-points,  for  which  the  standard  deviation  and  max¬ 
imum  error  in  comparison  with  the  triangulated  terrain  is  com¬ 
puted  and  is  compared  with  defined  maximum  values. 


Culture  Transformation 

The  DLMS  faco  characteristics  arc  p'ojected  on  the 
triangulated  terrain  so  that  a  faco  subdivision  is  performed. 
The  attributes  of  these  faces  can  be  gained  from  the  Informa¬ 
tion  on  the  material  category  and  the  Identification  code.  For 
point  features,  only  the  positions  will  be  transmitted  during 
transformation.  During  the  interaction  it  must  be  decided 
whether  a  line  feature  describes,  for  example,  a  river  such  that 
a  long  stretched  face  has  to  be  projected  into  the  terrain,  or 
only  a  bridge  has  to  be  shown,  which  is  called  ud  from  the 
model  library. 

Further  data  on  texturing  and  curved  shading  are  allocated 
interactively. 

interaction 

In  order  to  correct  the  deficiencies  in  the  digital  source 
material  and  those  which  have  been  introduced  during  the 
transformation  program,  it  is  necessary  to  perform  interaction 
during  the  different  phases  of  the  data  base  generation.  One 
starts  with  non  digital  source-  material,  as  for  example 
geodetic  charts,  air-photographs,  etc.  The  information  of 
those  data  mediums  are  brought  into  the  data  bases  by  uso  of 
the  interactive  station. 

The  hardware  of  the  interactive  station  consists  of  a 
graphics  terminal,  a  digitizing  tablet,  and  a  hard-copy  unit.  The 
source  material  is  positioned  on  the  digitizing  tablet  and  is 
scanned  by  means  of  a  digitizer.  The  digitizing  process  is  trac¬ 
ed  on  the  gtaphics  display.  A  cursor  defines  the  position  of  the 
digitizer  unit.  With  a  menue  attached  to  the  digitizing  tablet,  or 
by  means  of  the  alphanumeric  keyboard  of  the  graphics  ter¬ 
minal,  additional  information  can  be  put  in.  The  interactive  sta¬ 
tion  is  supported  by  a  complex  software  package  which  allows 
one  to  select  certain  data  as  well  as  to  change  or  input  new 
data.  This  software  package  limits  the  menue  effort  to  a 
minimum  and  is  user  friendly. 


Correction  o'  DLMS-data 

Before  the  transformation  process  starts,  it  has  to  be  en¬ 
sured  that  the  available  DLMS  data  are  in  accordance  with  the 
convention  of  the  DLMS  specifications.  Experience  has  shown 
that  the  number  of  errors  which  have  to  be  corrected  interac¬ 
tively  varies  according  to  the  care  with  which  the  data  was  col¬ 
lected.  In  this  phase,  single  characterstics  can  be  changed  or 
completely  new  characterstics  can  be  added  to  the  DLMS  for¬ 
mat. 

Correction  of  the  Online  Data  Bases 

All  necessary  information  which  can  not  be  gained  from  the 
DLMS  data  bases  must  be  added  interactively  after  the 
transformation  has  been  performed.  These  are  primarily  the 
lines  of  communication  which,  to  a  certain  extent  ,  are  not  ex¬ 
istent  In  the  DLMS  data  bases.  Further  data  on  new  faces  or 
models  has  to  be  added  where  the  transformation  program 
shows  only  non-identified  lines  or  point  features.  Addibonally, 
more  models  can  be  superimposed.  Finally  at  this  stage,  in¬ 
teraction  is  necessary  in  order  to  allocate  specific  alt; ibutes 
such  as  color,  texturing,  or  curved  surface  shading  as  well  as 
the  creation  of  universal  features. 

Generation  of  three-dimensional  Objects  and  Models 

For  the  model  library  a  complex  store  of  objects  and  models 
is  created.  This  is  necessary  in  order  to  describe  the  different 
culture  features  In  the  terrain.  The  basic  forms  of  simple  ob¬ 
jects  are  read-in  on  the  digitizing  tablets  from  construction 
drawings.  These  ob|ects  can  be  varied  by  simple  geometrical 
operations  like  scaling,  mirror  Inversion,  rotation,  etc.  Com¬ 
plex  models  can  be  built  upTnteractlvely  from  different  ob¬ 
jects.  The  German  Airforce  and  Navy  require  a  complex  model 
library.  The  models  Include  a  drilling  platform,  mine  shaft 
superslructure.  Chemical  plant,  coke  plant,  refinery, 
transformer  yard,  processing  industries  with  different  shapes 
of  roofs,  scrap  yard,  rotating  cranes,  railway  stations,  different 
types  of  bridges,  open-ended  stadiums,  family  houses, 


castles,  residences,  hospitals,  different  types  of  church 
towers,  airport  control  tower,  runways,  aircraft  parking  areas, 
taxi  ways-  drydocks.  navigation  light  ship,  lignt  ho  ises,  etc. 

Data  Base  Verification 

The  completed  data  bases  are  demonstrated  on  the 
graphics  display  of  the  Data  Base  Generation  System  The 
geometrical  construction  of  the  data  bases  can  be  evaluated. 
However,  all  attributes  are  only  numerically  indicated  (code 
tables).  In  order  to  get  a  complete  impression  of  the  data 
bases  with  respect  to  texturing  curved  surface  shading,  color 
and  the  three-dimensional  relationships,  it  is  necessary  to 
demonstrate  them  on  an  actual  CGIVS.  By  performing  this  ver- 
fication,  the  data  base  will  also  be  evaluated  with  respect  to 
the  dynamic  appearance.  The  verification  .Tight  result  in  the 
need  for  corrections,  which  can  be  accomplished  in  one  of  the 
above  mentioned  steps. 

Verification  of  data  bases  on  the  real  CGIVS  is  row  used  on¬ 
ly  temporarily  In  order  to  prevent  use  of  the  simultors  at  the 
souadrons  tor  data  base  generation,  it  is  planned  to  extend  the 
Data  Base  Generation  System  by  adding  a  non-real-time  Com¬ 
puter  Generated  Image  System. 

German  Data  Base  Visual  Requirements 

For  the  time  being  the  German  Airforce  and  Navy  require  the 
following  complete  data  bases. 


-  Transport  aircraft 
- 1  anker  aircraft 

Typical  kinds  of  ships 

-  Ground  vehicle  (heavy) 

-  Ground  vehicle  (medium  heavy) 


By  the  mid  1980's  the  German  Airforce  and  Navy  will  create 
their  own  data  bases  in  a  common  data  base  generation 
center  using  the  equipment  here  described. 


RADAR  SIMULATION  SYSTEM  DATA  BASE  GENERATION 

The  Data  Base  Generation  System  also  makes  it  possible  to 
generate  data  bases  for  the  Digital  Radar  l.andmass  Simula¬ 
tion  (DRLMS)  System.  In  order  to  gam  a  DRLMS  data  base,  a 
transformation  of  United  States  Defense  Mapping  Agency 
(DMA)  source  data  is  performed.  The  system  allows  interactive 
modification  of  the  DMA  data  bases  or  the  generation  of  new 
data  bases  from  topographic  charts.  The  same  hardware  is  us¬ 
ed  for  generat'on  of  both  the  radar  and  visual  data  bases. 
Verification  of  completed  data  bases  is  performed  on  an  ac¬ 
tual  DLRMS.  at  least  for  the  time  being. 


a  Navigational  area  o<  upper  Bavaria,  including  an  airforce 
base  and  a  NATO-slandard  bombing  range 

b.  Sea  area  with  a  coastline 

c.  Seaport 

d.  Complex  model  library  -  The  generation  of  simple  objects 
and  models  has  been  mentioned  before.  Me  re  complex 
models  to  be  designed  are. 

-  Fighter  aircraft 


Automatic  Transformation 

The  on-line  DRLMS  data  bases  are  produced  from  the 
source  material  by  means  of  a  translator  program  wherein  the 
grid  format  is  automatically  transformed  into  a  compressed 
vector  format.  The  translator  program  consists  of  two  parts: 
the  terrain  transformation  ana  the  culture  transformation. 
Both  processes  are  performed  separate1'/  and  the  information 
is  later  combined  to  achieve  the  on-line  DRLMS  data  bases. 
The  transformation  program  uses  a  variable  compression 
technique  with  the  following  characteristics: 


MAPS,  T  RAN^P  ARf  NO  If4',  ST  [  r<£  OPHOT  OGRAPHS 


234 


i 

i 


The  terrain  is  approximated  by  plane  faces  whm.h  are  con 
structed  with  lines  (vectors).  One  vector  indicates  a  change  in 
the  elevation.  The  number  of  vectors  which  are  necessary  to 
model  a  given  terrain  is  dependent  on  the  roughness  of  the  ter¬ 
rain  and  the  required  accuracy  of  the  image. 

Isolated  objects  such  as  towers  and  power  pylons  are  defin¬ 
ed  as  point  targets.  Point  targets  can  be  identified  by  their 
radius,  their  height  above  ground,  and  by  their  directional  or 
nondirectional  reflectivity  characteristics. 

Characteristics  with  length  dimension  ',e.y..  bridges,  streets, 
etc  )  are  def.ned  by  line  segments  Height  above  ground  and 
reflectivity  data  can  be  added. 

Large  area  features  like  lakes,  irregulariy  shaped  buildings, 
•ate.,  can  be  defined  by  mulitple  line  features.  Height  and 
reflectivity  data  are  added.  Each  data  point  ihat  is  stored  in¬ 
cludes  information  concerning  position  and  height  in  respect 
to  the  foregoing  value.  Therefore  all  of  these  values  can  be 
defined  ns  vectors.  In  comparison  to  code  techniques  with 
constant  grid  distance,  the  variable  compression  technique 
results  in  a  substantial  saving  of  storage  capacity  This  allows 
a  much  better  and  more  realistic  image  in  systems  of  com- 
pariable  storage  capacity. 

Interactive  Data  Base  Modification 

After  navmg  performed  the  first  step  of  transformation,  the 
data  base  can  be  improved  interactively  In  addition,  there  is 
the  possibility  of  modifying  existing  data  bases  by  subtracting 
nr  adding  parts  or  changing  the  reflectivity  Dimensional 
characteristics  can  be  added  and  features  can  be  changed  in 
respect  to  length,  width  and  height. 

Interactive  Data  Base  Generation 

An  additional  program  of  the  DBGS  allows  data  base 
generation  without  using  Defense  Mapping  Agency  Data  In¬ 
teractively  generated  terrain  data  are  formatted  in  a  way  that 
they  are  compatible  with  data  received  from  the  automatic 
transformation  program  In  addition,  interactive  generation 
and  formatting  of  cu'ture  feature  data  car  be  performed. 

Verification  and  Requirements  for  Radar  Data  Bases 

The  data  bases  gat. ted  from  the  DBGS  process  can  be 
verified  by  means  of  the  actual  DRLMS.  The  same  applies  for 
modified  radar  data  bases  Using  the  DRLMS.  the  realistic 


dynamic  appearance  is  evaluated.  However,  in  order  to  use  the 
actual  DRLMS  only  far  naming  purposes,  the  Data  Base 
Generation  System  will  be  extended  by  computer  peripherals 
so  that  the  verification  process  can  be  pe-formed  with  the 
Data  Base  Generation  System. 

At  present,  a  realtime  DLMS  data  base  is  required  for  upper 
Bavaria 


Visual  and  Radar  Data  Base  Correlation 

Visual  data  bases  as  well  as  radar  data  bases  are  developed 
by  indi  dry  with  the  assistance  of  the  user.  Figure  4  shows  the 
data  base  generation  for  Ctuvs  and  DRLMS  systems.  By  us¬ 
ing  identical  source  material,  correlation  o4  the  displays  for 
both  simulation  systems  is  possible. 
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ABSTRACT 


''Maintenance  training  simulators  have  proven  to  afford  equal  or  superior 
training  at  a  lower  life  cycle  cost  than  actual  equipment  trainers  when  teaching 
troub  leshoot  ing  brsed  on  front  panel  indications,  failure  symptoms  and  some  in- 
drawer  visual  indicators.  The  purpose  of  the  study  was  to  determine  the  effects 
of  two-dimensional  and  three-dimensional  fidelity  of  simulation  and  three  levels 
of  reduced  accessibility  to  test  points  during  training,  on  student  trouble¬ 
shooting  performance  while  locating  faults  at  the  component  level.  A  total  of 
186  students  were  observed  and  tested  in  the  ET  Splice  modules  of  a  Navy  Basic 
Electricity  and  Electronics  course.  Conclusions  are  drawn  about  the  relative 
training  effectiveness  of  simulated  and  actual  boards  and  recommendations  are 
made  in  selecting  active  test  points  on  simulated  printed  circuit  boards.. 


INTRODUCTION 

Over  the  last  few  years,  computer  simula¬ 
tion  maintenance  trainers  have  made  significant 
inroads  against  actual  equipment  trainers 
(AET's)  in  hands-on  electronic  maintenance 
training.  Simulators  have  proven  to  provide 
equal  or  superior  training  at  a  lower  life 
cycle  cost  when  teaching  troubleshooting  based 
on  front  panel  indications,  failure  symptoms 
qnd  some  in -drawer  visual  indicators. 

However,  in  the  area  of  hands-on  trouble¬ 
shooting  to  thp  component  level,  the  relative 
cost-effectiveness  of  AET's  versus  simulation 
trainers  is  not  clearly  understood.  Actual 
equipment  trainers  are  a  higher  fidelity  simula¬ 
tion  of  the  field  equipment  and  theoretically 
should  provide  better  transfer  of  training. 
However,  high  AET  purchase  costs  and  lower 
reliability  leads  to  high  life  cycle  costs. 
Trainers  generally  have  a  lower  life  cycle 
cost,  but  these  savings  are  accompanied  by  a 
reduced  fidelity  of  simulation;  especially  a 
reduced  number  of  test  points.  Simulation 
engineers  indicate  that  if  all  test  points  on'ff 
circuit  board  (50-100  points)  are  simulated, 
the  complexity  of  modeling  the  correct  test 
equipment  readings  for  each  failure  at  every 
point  becomes  prohibitive.  ' 

Another  difference  between  AET’s  and 
simulation  trainers  is  that  trainers  may  uti¬ 
lize  a  photograph  of  a  circuit  board  with  test 
points  available  in  appropriate  locations.  The 
training  effects  of  this  reduced  fidelity  of 
simulation  have  not  yet  been  determined. 

The  general  assumption  is  that  AET  is  more 
costly  and  effective  in  training  for 
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troubleshooting  to  the  component  level,  while 
simulation  trainers  are  less  expensive  and  less 
effective  due  to  the  limited  number  of  test 
points  and  reduced  visual  fidelity.  The  ques¬ 
tion  is  which  one  is  more  cost-effective. 
Engineers  can  estimate  the  cost  of  a  trainer 
for  various  numbers  of  simulated  test  points 
and  visual  representation  based  on  previous 
experience.  The  question  remains  as  to  the 
relative  effectiveness  of  a  trainer  depending 
on  the  fidelity  of  simulation  and  number  of 
test  points  simulated.  The  purpose  of  this 
study  was  to  determine  the  transfer  of  training 
to  actual  equipment  derived  from  training  on 
modified  printed  circuit  boards  with  varying 
numbers  of  simulated  test  points  represented 
photographically  and  in  three  dimensions. 

METHOD 


Initial  Data  Acquisition 


Initial  data  were  collected  to  determine 
the  points  most  frequently  probed  by  Oasic 
Electricity  and  Electronics  ( BE&E )  students. 

These  initial  data  were _ required  in  order  to 

select  the  points  to  be  exposed  during  the 
experimental  phase  of  the  study. 


Students  were  categorised  as  high,  medium 
and  low  proficiency  levels  based  on  prerequi¬ 
site  course  completion  time.  Trainees  were 
observed  during  normal  coursework  and  trouble¬ 
shooting  lessons.  No  changes  were  made  in  the 
current  curriculum  except  for  the  additional 
troubleshooting  session  at  the  researcher’s 
table.  The  observation  of  troubleshooting 
behavior  resulted  in  the  following  data: 


l)  number  of  trainees  probing  eavh  test 
po  int 
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Iho  board  modification  in  Figure  2 
simulates  the  effect  of  varying  numbers  of  test 
points  on  a  photographic  simulation  of  a  PC 
board.  A  photograph  of  the  PC  board  was 
mounted  over  the  actual  board.  Test  points 
were  created  by  placing  a  hole  in  the 
photograph,  projecting  the  copper  wire  through, 
and  coating  it  with  varnish.  Test  points  were 
then  made*  accessible  by  cutting  away  the 
coating  in  the  same  manner  as  above. 

Table  L  indicates  the  number  of  test 
points  made  accessible  on  the  FM  Radio  Second 
IF  Amplifier  Board.  Faults  were  grouped  togeth- 
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*NOTl\  :  Using  Half-Split  Technique 
Sub joe ts 

Test  subjects  were  trainees  in  the  RT 
Splice  modules  of  a  Navy  BF.&E  course.  A  total 
of  18b  trainees  were  included  in  data 
co 1 1 ec  t  ion . 

Experimental  Design 

A  two-way  analysis  of  variance  design  (see 
Figure  ?)  was  used  for  the  main  independent 

variables  of  fidelity  (actual  boards  vs 
photographic  boards)  and  three  levels  of  test 

point  availability  (100%,  67%,  33%),  with  a 

control  group  (unmodified  boards). 

Three  different  circuit  boards  were  uti¬ 
lized  in  the  study;  an  FM  Rad  io  First  IF 
Amplifier  board,  FM  Radio  Second  IF  Amplifier 
board  and  a  Power  Supply  board.  Each  had  three 
fault  group  types.  Trainee  proficiency  levels 
were  matched  for  each  treatment  condition.  The 
dependents-/ var  iab  les  were  number  of  test  points 
probed,  time  to  probe,  and  number  of  trips  to 
the  learning  supervisor  before  fault 
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localisation.  In  addition  to  the  main  ex  per  i- 
metnal  effects,  the  troubleshooting  logic  used 
by  trainees  was  examined.  The  logic  was 
examined  for  differences  by  'experimental 
treatment,  as  well  as  analysis  of  what  types  of 
strategies  were  used  most  consistently  and 
o f  fee C i ve l v  . 


*Each  coll  balanced  with  3  levels  of  pro¬ 
ficiency  and  3  fault  groups. 

Figure  3  Experimental  Design 

P  rocedu res 

When  the  subject  trainees  were  ready  for  a 
practice  session  on  one  of  the  boards  used  in 
the  study,  they  were  assigned  to  the  research 
station.  The  experimenter  gave  the  trainee  a 
pro-  faulted  board  modified  to  one  of  the  seven 
treatment  conditions.  Subjects  proceeded  to 
troubleshoot  the  board  and  take  their  exercise 
sheets  to  the  school's  learning  supervisor  for 
grading.  This  step  was  repeated  with  an  identi¬ 
cal  board  and  treatment  condition,  but  a  differ¬ 
ent  fault.  When  the  learning  supervisor 


determined  the  subject  had  mastered  the  board, 
the  trainee  was  given  an  unmodified  board  to 
troubleshoot.  This  test  was  the  criterion 
performance  to  measure  transfer  of  training  to 
actual  equipment  after  practice  on  modified 
boards . 

During  the  troubleshooting  of  both  mod¬ 
ified  and  unmodified  boards,  the  experimenter 
recorded:  the  number  of  probes,  probing  time, 

subject  comments  and  all  other  applicable  data. 
These  data  were  then  analyzed  to  determine  the 
training  effects  of  simulation  fidelity  and 
test  point  availability  on  troubleshooting 
behavior  using  actual  equipment. 

RESULTS 

Probes  and  Time 

At  the  time  this  paper  was  prepared, 
analyses  were  completed  for  the  FM  Radio  Second 
IF  Amplifier  board  only.  Results  for  the  other 
boards  will  be  reported  at  a  later  date. 

The  primary  measures  of  effectiveness  were 
number  of  probes  and  time  required  to  locate 
the  fault.  Figure  4  exhibits  the  effects  of 
percent  points  available,  fidelity  and  student 
proficiency  level  on  these  two  measures  during 
ci  iter  ion  tests  on  actual  equipment. 

Students  tended  to  probe  fewer  times  and 
locate  the  fault  in  less  time  when  trained  on 
the  board  with  67X  of  the  points  accessible. 
However,  this  difference  was  not  statistically 
significant.  Students  tended  to  probe  fewer 
times  and  locate  the  fault  in  less  time  when 
trained  on  unmodified  boards  as  opposed  to  2D 


Figure  U  Probca  and  Time  By  Experimental  Condition 
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T r o ub l os h oot in g  Log i c 

!'ho  specific  points  probod  by  students 
dor  ini'  per  form.ince  tests  were  analvzed  in  order 
provide  guidance  to  engineers  in  selecting 
which  points  students  arc  most  likely  to  probe. 
Analyses  addressed  troubleshooting  logic  and 
c h a r a c  t e r  L s  t i c s  of  points  p r o be d . 


Symptomn-  Testing  based  on  equipment 

t i c  symptoms 

Random  No  lot;  teal  semtenre  oF  nui 


Comb i na  - 
t  i  on 


No  logical  sequence  of  tests 
Use  of  two  or  more  strategies 


Since  the  dominant  strategy  is  random  probes, 
no  rules  can  be  recommended  for  selecting 
active  test  points  based'  on  troubleshooting 
Strategy. 


Table  3 

Troubleshooting  Strategies  Utilized 


J imf>s  Utilized 
l 


Strategy 

Half-Split 

Linear  I/O 

Linear  Tracing 

Rel iab i I i t  y  Tes t ing 

Symptomat ic 

Random 

Comb  inat ion 


An  analysis  of  characteristics  of  the 
printed  circuit  board  components  tested  by 
students  was  conducted  in  order  to  determine 
whether  the  characteristics  of  components 
affect  the  frequency  with  which  they  are 
probed.  Table  4  contains  the  results  of  this 
analvsis. 


Test  Point  JLrobes 
Hy  Component  Characteristics 
Expected  Versus  Observed  Frequency 


The  sequences  of  probes  for  each  of  the 
performance  tests  were  analyzed  to  determine 
which  troubleshooting  strategies  were  utilized 
by  the  students.  Results  appear  in  Table  3. 
Definitions,  of  those*  strategies  are: 

Half-Split  Testing  the  midpoint  bet  wee:  n 
good  and  bad  signal  until 
fault  located 

Linear  I/O  Testing  output  of  each 
c ire u  it 

Linear  Testing  components  sequen- 

Tracing  tially  until  faulty  signal 

fo  und 

Reliability  Testing  least  reliable 

Testing  untested  component 


Charac¬ 
ter  is  t  ic 

Type 

Expected 

Probes 

Observed 

Probes 

0i  f  fer- 
once _ £? ) 

E  lec- 
t  ron  ic 

Input 

201 

125 

-38 

Locat ion 

Midd  le 

100 

210 

+  101 

Output 

297 

346 

+  16 

Other 

3583 

3509 

-2 

Elec¬ 
tron  ic 

Input 

vO 

OO 

458 

-8 

C i rcu  it 
Locat  ion 

M i dd  le 

100 

201 

+  101 

y-r-  rqp 

Jr 4.  ¥  i  '\r 

ABLE  Cl 
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Table  4 
( C  on  t  '  d  )” 

Test  Point  Probes 
By  Component  Characteristics 
Expected  Versus  Observed  Frequency 


Charac- 
_ter  is  t  ic 

Type 

Expec  ted 
Probes 

Observed 

Probes 

Di  f  fer 

ence 

Output 

493 

462 

-7 

Other 

1035 

3060 

-.8 

Phv s i ca  l 

Edge 

1793 

2025 

+ 1  3 

Local  ion 

Midd  le 

29  7 

514 

+  73 

Other 

2091 

164  2 

-21 

Si,e 

Large 

f>98 

970 

+  38 

Smn  1  t 

3433 

3211 

-7 

board : 

Electronic  Midpoint  of  Board 
Board  Outputs 

Electronic  Midpoint  of  Circuit 
Test  Points  Associated  With  Large 
Component  s 
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"If  students  test  components  without  regard 
t'  their  characteristics,  then  the  expected 
proportion  of  probes  associated  witii  each 
component,  wi  ]  l  he  equal  to  the  proportion  of 
test  points  associated  with  that  type  of  compo¬ 
nent.  A  Chi  Square  Analysis  of  expected  versus 
observed  probe  frequency  found  the  differences 
to  be  significant  (p  >  0.01 )  for  all  component 
elm  rue t  e  r i s  r i c s  . 


Students  test 

the 

mi dpo  i n t 

in  the  board 

circuitry 

as  the  first  step  in 

a  half-split 

Lee  hn  i  q  lie  , 

but  they 

do 

not  continue  this  proee- 

du  re  tin  t  i  1 

the  fan 

Ity 

componen  t: 

is  located. 

The  board 

output  is 

ten 

ited  more  often  than  the 

input,.  The  physic* 

tl 

midpoint  was  near  the 

o  l  ee  t  run  i.  c 

mi  dpo  in  t 

and 

shows  the 

same  trends. 

Once 

a  circuit 

on 

the  board 

is  suspected 

of  being 

fan  1 1  y  , 

the 

s  t  uden t 

predominant ly 

probes  the  midpoint  of  the  circuit,  as  opposed 
fo  t ho  input  or  output.  Finally,  students  are 
mor.>  likely  to  test  a  large  component  than  a 
sum  !  I  one  . 
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CONCLUS  LONS 

Basic  electronics  training  on  circuit 
boards  with  reduced  physical  fidelity  and 
reduced  test  point  accessibility  is  equal  to  or 
superior  to  training  on  unmodified  actual 
circuit  boards.  Student;:  trained  on  two- 

dimensional  hoards  with  2.  75  times  as  many  test 
points  available  ns  required  had  the  best 
overall  t  rotib  leshoot  ing  performance  during 
t*s  r  ing. 

0 tiier  than  the  minimum  points  specifically 
required  to  detect  a  fault,  the  following 
points  should  bo  accessible  on  a  simulated 
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THE  INTEGRATION  OF  VIDEODISC,  CAI,  AND  3D  SIMULATION 
POR  SKILLS  TRAINING 
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Grumman  Aerospace  Corporation 


ABSTRACT 


"Training  in  maintenance  skills  has  become  increasingly  more  important  as  the  cost  of  replacemenf 
parts  and  expenditures  for  maintenance  personnel  have  risen.  More  effective  .md  efficient  skills  train¬ 
ing  has  been  identified  as  a  means  to  limit  costs  through  fewer  false  repairs,  shorter  down  time,  and 
decreased  numbers  of  personnel  required  for  maintenance.  Advocates  have  championed  various  systems  and 
devices  for  this  training,  to  include  in  different  formsV  actual  equipment,  flat  panel  simulators,  three 
dimensional  simulators,  videodisc,  and  computer  assisted  instruction.  -^This  paper  discusses  the  inte¬ 
gration  of  interactive  videodisc,  computer  generated  images,  and  three-dimensior al  simulation  in  a  total 
system  concept  for  maintenance  skills  training.  Different  types  of  maintenance  skills  are  identified, 
along  with  the  methods  and  techniques  for  training  those  skills.  Implementation  of  the  methods  and 
techniques  in  an  integrated  system  is  presented,  to  include  the  means  for  providing  modeling,  drill  and 
practice,  cueing  and  prompting,  feedback,  and  evaluation.  Two  different  systems  are  identified, 
rationale  for  the  differences  is  provided,  and  the  advantages  each  has  in  intended  use  is  specified. 


INTRODUCTION 

The  training  of  maintenance  personnel  has 
alway  been  of  critical  importance.  Training  pro¬ 
blems  have  become  even  more  acute  in  the  recent 
era  of  runaway  technological  advances  that  las  re¬ 
sulted  in  increasingly  complex  equipment. 
Additionally,  the  costs  of  replacement  parts  and 
expenditures  for  maintenance  personnel  have  risen. 
Of  particular  concern  to  those  involved  in  main¬ 
tenance  training  is  the  teaching  of  effective  and 
efficient  troubleshooting,  as  a  means  cf  limiting 
costs  through  fewer  false  repairs,  shorter  down 
time,  and  decreased  numbers  of  maintenance 
personnel.  In  a  time  of  decreasing  training  bud¬ 
gets  that  necessitate  shorter  maintenance  training 
courses  and  increasing  student/instructor  ratios, 
it  is  i-  perutivp  that  the  methods  and  media  used 
for  training  be  maximally  effective. 

This  paper  addresses  the  some  of  the  diffi¬ 
culties  associated  with  maintenance  training,  it 
first  addresses  the  types  cf  skills  required  of 
the  maintenance  technician.  It  then  discusses 
maintenance  training  from  the  standpoint  of  the 
lea^niny  curve  and  talks  to  the  media  and  methods 
that  have  traditional ly  be**n  used  to  teach  the 
trainee  at  each  stage  of  learning,  finally,  it 
discusses  the  use  of  an  integrated  system  concept, 
made  possible  by  recent  technological  advances. 
This  technology  has  been  applied  to  maintenance 
training,  and  this  paper  describes  two  systems 
that  have  beer  developed  to  apply  the  concept. 

MAINTENANCE  SKILLS 

Lquipment  maintenance  requires  a  diversity  of 
skills,  which  range  from  simple  tasks  such  as  ro¬ 
tating  a  switch  to  very  difficult  tasks  such  as 
interpreting  and  following  schematics  in  the  pro¬ 
cess  of  tracing  signal  paths.  Both  cognitive  and 
psychomotor  behaviors  arc  required  of  the  main¬ 
tenance  technician  to  perfor-  these  tasks.  Table 
1  illustrates  some  of  these  behaviors. 
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TABLE  1 

Sample  Maintenance  Behaviors,  "isu  i  le  Maintenance 
(Ac  up  ted  from  AMTESS  Phase  1  Final  Report)' 
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It  is  evident  that  there  are  many  enabling 
skills  and  knowledges  that  the  trainee  must  have 
before  he  begins  to  acquire  the  specific  mainten¬ 
ance  skills  listed  in  the  table.  The  enabling 
skills  include  such  things  as  terminology,  com¬ 
ponent  locations,  functional  system  knowledge,  _ 
use  of  technical  materials,  and  basic  electronics. 
An  effective  and  efficient  maintenance  technician 
must  possess  both  of  these  types  of  skills. 

MAINTENANCE  TRAINING 

fhe  training  of  maintenance  skills  Ins  been 
a  difficult  proposition.  From  the  standpoint  of 
a  training  system,  the  student  must  be  taught 
each  of  the  many  types  of  maintenance  skills. 

Each  type  of  skill  must  be  taught  in  a  way  that 
is  consistent  with  its  nature.  In  addition,  the 
synergistic  nature  of  the  maintenance  environment 
requires  that  the  student  be  taught  to  use  all  of 
these  skills  as  an  integrated  whole. 

Learning  Utaqes 

The  acquisition  of  knowledge  is  almost  always 
a  gradual  process.  As  Myers^  notes,  the  classi¬ 
cal  learning  curve  can  be  divided  into  three 
learning  stages:  acquisition,  consolidation,  and 
evaluation.  In  the  acquisition  stage,  the  student 
gains  an  initial  familiarity  with  the  subject 
matter,  in  the  consolidation  stage,  he  gains 
criterion  competency.  His  ability  to  reform  is 
tested  in  the  evaluation  phase,  and  administrative 
deciS'Ons  are  made  on  the  basis  of  his  performance. 

The  application  of  this  staged  process  in 
maintenance  training  is  clear.  The  student  should 
first  be  guided  in  the  acquisition  of  basic 
knowledge  and  skills  using  techniques  that  are 
appropriate  for  mis  type  of  learning.  This  is 
followed  by  practice  that  requires  the  use  of 
these  sxill  in  a  realistic  mamer.  "he  cues, 
prompts,  helps,  and  remediation  used  during  this 
practice  should  gradually  be  reached  until  the 
student  is  able  to  perform  at  a  criterion  level 
using  only  the  materials  which  will  be  available 
to  him  on  the  job. 

In  looking  at  equipment  naintenance  holisti¬ 
cally,  it  appears  that  the  acquisition  stage 
addresses  the  enabling  skills  -and  knowledge  dis- 
cussed  above.  These  skills  a”J  knowledges  are 
primarily  cognitive  in  nature,  though  they  are 
Certain  to  include  some  psychomotor  elements  as 
well.  These  tyoeS  of  behavior  can  usually  be 
taught  using  low  fidelity  representations  such  as 
textual  descriptions,  line  drawings,  photographs, 
motion  pictures  and  other  two-dimensional  media. 

During  the  equation  stage,  tne  student  "'us t 
be  able  to  perform  complex  cognitive  and  psycho¬ 
motor  behaviors.  The  emphasis  in  this  stage  is 
on  the  student's  psychomotor  activities,  since 
his  cognitive  processes  are  ncrmal'y  manitested 
in  his  actions.  Thus,  testing  the  student's 
ability  requires  the  use  or  so^e  higher  fidelity 
three-dimensional  medium,  such  as  actual  equip'"t”t 
or  a  simulator. 

The  consolidation  stage  provides  a  transition 
between  the  primarily  cognitive  acquisition  stage 
a*  d  tiic  pri-uri  ly  psychomotor  evaluation  stage. 
Although  the  student  learns  about  maintenance  in 
the  jcqosit  ion  stage,  it  is  in  the  consoli¬ 


dation  stage  that  he  actually  learns  to  do  main¬ 
tenance.  It  is  here  that  he  will  gain  knowledge 
of  the  synergistic  aspects  of  the  job  he  is  to 
perform.  The  gradual  fading  of  cues,  prompts, 
etc.,  inherent  in  this  stage,  necessitates  the 
concurrent  use  of  both  low  fidelity  two- 
dimensional  media  and  high  fidelity  three- 
dimensional  media. 

Methods  and  Techniques 

Various  methods  and  techniques  have  proved  to 
facilitate  training  of  maintenance  skills.  Media 
selected  for  training  must  be  capable  of  incorpo¬ 
rating  these  training  aids.  Examples  include 
guided  practice,  modeling  followed  by  immediate 
application,  and  realistic  representations  of  mal 
functioning  equipment.  In  addition,  those  proven 
methods  and  techniques  common  to  all  successful 
training  should  be  available.  This  would  include 
capabilities  such  as  self-pacing,  remediation, 
cueing  and  prompting,  and  branching 

TRADITIONAL  TRAINING  MEDIA 

Media  traditionally  used  for  maintenance  train 
ing  have  met  the  requirements  of  the  three  stages 
of  maintenance  training  with  varying  degrees  of 
success.  Successful  training  has  occurred  when 
well-designed  instruction  has  been  delivered 
using  media  appropriate  for  the  learning  stag  in 
which  they  have  been  used.  Pr^nt  and  other  two- 
dimensional  media  have  been  successfully  employed 
most  often  when  used  in  the  acquisiton  stage. 
Actual  equipment  and  three-dimensional  simulators 
have  been  successfully  used  tor  evaluation. 
Appropriate  integration  of  the  use  of  2D  and  3D 
media  has  resulted  in  some  success  in  the  con¬ 
sul  idaticn  stage,  Unfortunately,  this  inte¬ 
gration  has  historically  been  difficult  to 
achieve. 

Mcquisition  Stage 

A  wide  variety  of  two-dimensional  media  have 
been  used  effectively  in  the  acquisition  stage. 
These  media  include  various  print  materials, 
e.g.,  textbooks,  programmed  texts,  and  workbooks; 
photographic  materials,  e.g.,  photographs ,  si  ides 
movies,  and  video;  art  work;  ana  audio  materials. 
Perhaps  the  most  powerful  2D  medium  in  existence 
at  present  is  the  relatively  new  tecnnology  whicn 
combines  the  use  of  interactive  videodisc  and 
computer  graphics  in  a  single  device. 

evaluation  stage 

Historically,  the  preferred  3D  "’edi-/"  for  tne 
evaluation  stage  has  been  actual  equipment 
trainers  (Alls).  Limitations  inherent  in  ^sing 
iLTs  have  resulted  in  the  growing  use  of  si"u- 
lators.  These  simulators  have  inclosed  vario-s 
levels  of  phy  ical  and  functional  fidelity, 
ranging  from  low  fidelity  flat  panel  simulators 
to  high  fidelity  3D  replications  of  actual  eq-ip- 
•"ent.  The  degree  of  fidelity  required  is  a 
function  of  tne  tasks  to  be  evaluated. 

LoniJlUation  btage 

The  consol idat ion  stage  requires  ^Se  ot  both 
DL  and  3D  nedia.  Any  cf  the  "edia  identi- 
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fled  above  can  be  used  effectively  in  this  stage. 
But,  it  is  apparent  from  the  requirements  for 
this  stage,  as  discussed  above,  that  all  of  the 
media  used  in  this  stage  must  work  together  as  a 
unified  whole.  This  requires  the  use  of  some 
intelligent  controller  to  coordinate  the  student's 
interaction  with  all  of  the  media  employed.  The 
use  of  an  instructor  as  controller  is  inefficient, 
in  terms  of  both  .  ime  and  money.  When  administra¬ 
tive  factors  dictate  a  high  student  to  instructor 
ratio,  the  training  can  be  ineffective  as  well. 

Recert  advances  in  technology  allow  extensive 
use  of  computers  in  the  consolidation  phase.  Com¬ 
puters  are  used  to  control  30  simulators,  to  con¬ 
trol  videodisc  players,  and  to  control  other 
peripheral  equipment.  This  allows  the  use  of  the 
computer  as  the  intelligent  controller  in  the  con¬ 
solidation  phase.  This  mix  of  2D,  3D,  and  com¬ 
puter  as  cont  oiler  is  likely  to  be  the  most 
effective  means  of  providing  training  in  this  con¬ 
solidation  stage. 

INTEGRATED  SYSTEM  CONCEPT 

The  above  discussion  indicates  the  need  for 
a  system  which  integrates  the  use  of  both  2D  and 
3D  media.  Such  a  system  could  be  effectively 
used  in  all  three  learning  stages.  The  2D  portion 
used  alone  could  be  used  for  acquisition  appli¬ 
cations,  while  the  use  of  the  3D  by  itself  would 
suffice  for  evaluation.  In  addition,  the  inte¬ 
gration  of  the  use  of  the  two  in  various  propor¬ 
tions  would  make  it  a  potent,  cost-  and  training- 
effective  instrument  in  the  crucial  consolidation 
stage. 

Until  recently,  technology  has  not  been 
available  which  would  allow  the  integration  of 
2D  and  3D  media  into  a  irified  system.  Recent 
breakthroughs  have  allowed  the  Grumman  Aerospace 
Corporation  to  design  and  construct  two  such 
systems,  and  they  are  currently  under  operational 
evaluation. 


The  Army  J-taintenai.ee  Training  and  Evaluation 
Simulation  System  (AMTE3S)  is  a  generic  main¬ 
tenance  training  system  developed  for  the  U.S. 
Army,  Project  Manager  for  Training  Devices.  The 
version  of  AMTESS  developed  t\/  Grurian  *  3  con¬ 
sists  of  two  modules:  a  student  station  and  a 
3.  simulator  (See  Figure  1  ■ .  Tnese  ‘wo  modules 
can  be  used  as  independent  elements,  or  they  can 
work  together  as  an  integrated  system. 


3D  Mcdule,  3D  simulator  modules  can  be  either 
system  or  training  program  specific.  Math  models 
cause  each  module  to  react  as  the  real  equipment 
it  simulates  would  under  both  normal  and  deyraded 
conditions.  These  models  make  no  judgment  as  to 
the  correctness  of  student  actions.  The  3D  simu¬ 
lator  provides  the  technician  with  the  upportur.  i  ’  j 
to  manipulate  controls,  observe  indicators,  and 
troubleshoot  malfunctions  on  equipment  that  is 
physically  and  functionally  represeiitati  ve  of  the 
actual  equipm-ot.  Computer  control  of  the  3D  via 
a  microprocessor  urov^des  all  of  the  normal  and 
abnormal  indications  of  ali  components,  controls, 
and  indicators. 

iwo  different  prototype  3D  modules  have  been 
delivered  and  currently  are  under  evaluation.  An 
automotive  module  designed  to  train  generating  an. 
starling  system  troubleshooting  on  the  MlluA2 
self-propd led  howitzer  is  at  Aberdeen  Pruvin^ 
Ground,  Maryland  (See  Figure  2),  and  a  missile 
module  designed  to  train  troubleshooting  on  the 
transmitter  of  the  High  Rower  Illuminatoi  Radar 
of  the  HAWK  Missile  System  is  at  Fort  bliss, 
Texas.  (See  Figure  3). 
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2D  Module.  The  2D  module  can  be  used  in  a 
stand-alone  mode,  or  it  can  be  connected  to  any 
3b  module  for  tandem  operation  (See  Figure  4). 
Two-way  communication  between  the  student  and  the 
2D  is  accomplished  using  a  color  television 
monitor,  which  can  display  videodisc  and/or  com¬ 
puter  generated  images.  Student  2D  responses 
are  registered  through  a  touch  bezel  attached 
to  the  TV  monitor.  The  videodisc  player,  TV 
monitor,  and  touch  bezel  are  all  under  the  con¬ 
trol  of  a  single  microprocessor  with  dual  drive 
floppy  disk.  When  operating  ' n  tandem,  the  2D 
processor  also  controls  the  3D  simulator. 


Figure  4  AMT ESS  Student  Station 

The  2D  can  be  useo  for  various  types  of  in¬ 
structional  activities,  e.g.,  presentations, 
tutorials,  or  drill  and  practice.  Various  in¬ 
structional  techniques  can  be  used  *o  facilitate 
student  learning.  For  example,  modeling  of  ex¬ 
pert  behavior  on  the  2D  can  be  fo1 lowed  by 
'immediate  student  immitation  o *  the  modeled 
behavior  on  the  3D.  The  2D  can  monitor  student 
actions  on  a  continuous  basis.  It  can  branch 
to  provice  specific  guidance,  remediation,  and/or 
feedback  as  appropriate. 


The  Tactical  Jamming  Sustem  (TJS)  trainer 
tor  the  U.S.  Navy's  EA-63  aircraft  was  developed 
tor  the  Nava!  Air  Systems  Command,  and  has  been 
delivered  to  the  U.S.  Naval  *ir  Station,  Whidbey 
island.4  While  configured  somewhat  differently 
than  AMTESS,  the  TJS  contains  the  same  elements, 
i.e.,  videodisc,  computer  generated  graphics, 
student  response  touch  bezel,  computer  control, 
and  3D  simulation.  (See  Figure  5).  The  TJS 
trainer  is  weapon  system  specific,  and  although 
it  could  be  modified  for  other  applications, 
e.g.,  the  training  of  operator  procedures,  it 
was  developed  to  address  TJS  -cintenar.ee  train- 
•.  ig  requirements. 

SYSTEM  TRAIN  IN 3  CAPABILITIES 

Integrated  System  Trainers  (IjTs,'  arc  an 
answer  to  the  requirement  for  concurrent  use  of 
21  and  3D  edia  in  the  consol i cation  stage, 
having  both  2D  and  3D  capabilities,  they  also 
have  full  capability  for  use  in  t1-*.  acquisition 
and  evaluation  stages.  The  ability  to  deliver 
that  training  using  a  variety  of  "t'hods  and 


Figure  5  TJS  Maintenance  Trainer 

techniques  has  been  demonstrated  in  the  lessons 
developed  for  the  AMTESS  and  TJS  systems.  The 
following  paragraphs  describe  the  ways  in  which 
various  features  of  the  ISTs  have  been  used. 

The  heart  or  an  1ST  is  the  computer  in  the  2D 
which  controls  the  student's  interaction  with  <11 
elements  of  both  tne  2D  and  3D  modules.  It  meu i- 
tors  his  pi  ogress  and  provides  cues,  prompts,  in¬ 
struction,  knowledge  of  results,  and  help  to  the 
student  in  accordance  to  developed  lessen 
materials.  The  student  interacts  with  the  2D 
using  the  touch  bezel  attached  to  the  video  screen 
ana  his  manipulations  of  the  various  components  on 
the  module  are  reported  to  the  2D  computer  using 
the  communication  lines  between  the  modules. 

On  tne  2D  module,  viaeodisc  images  have  been 
used  to  store  instructional  materials  in  tne  for- 
of  textual,  graphic,  pictorial,  and  auditory  in¬ 
formation,  The  random  access  capability  of  the 
player  allows  the  computer  tu  call  up  either 
single  frame  or  motion  sequences  in  an  oroer  that 
is  appropriate  for  instructional  requirements. 

The  motion  capability  can  be  used  to  present 
audio,  visual,  or  audiovisual  sequences.  Two 
audio  channels  on  the  disc  permit  additional 
flexibility  of  use. 

Computer  generated  graphics  extend  the  capa¬ 
bility  of  the  videodisc,  and  provide  a  quick  and 
easy  way  to  update  lessons.  The  gen 'rati  or.  of 
th«e  computer  presentations  is  under  2D  computer 
control,  so  the  "essages  can  be  tailored  to  indi¬ 
vidual  circumstances.  Communication  between  the 
2D  and  3D  computers  allows  tne  syste'  to  call  up 
video  anc  computer  grapric  imaues  in  response  to 
student  actions. 

<.se  of  the  2J  co-belt r  as  the  executive  per¬ 
ils  the  two  parts  of  the  s/’-em  to  be  used  in  a'\> 
desired  proportion.  Tins  tlexibility  allows 
lesson  -aterials  to  be  tailc.-ud  to  the  specific 
requirements  of  each  leaning  stage. 

This  not  only  increases  instructor  reli¬ 
ability,  and  consistency  of  t'm’rg,  it  per- its 
the  instructor  to  use  ti~e  more  efficiently. 

This  allows  the  instructor  to  become  more  of  a  re- 
sd.rcu  -arager,  using  all  of  his  available  toe  Is, 
anq  additionally  frees  tne  instructor  to  spend 


time  with  those  students  who  would  best  benefit 
from  that  direct  interface. 

During  the  acquisition  stage,  videodisc 
images  and  computer  graphics  have  been  used  ex¬ 
tensively  to  provide  cues,  prompts,  and  reme¬ 
diation,  and  for  student  requested  nelp  sequences. 
Various  levels  of  help  have  been  made  available, 
with  .Alternate  paths  based  on  student  performance. 
Relatively  speaking,  very  little  use  of  the  3D 
module  has  been  <  ,ude  during  this  stage. 

During  the  consolidation  phase,  the  student 
is  weaned  from  cependence  on  the  2D.  Fairly  heavy 
use  is  made  of  the  student  station  early  in  this 
stage,  as  it  guides  his  interactions  with  the  3D. 
By  the  end  of  this  stage,  however,  the  student 
interacts  almost  exclusively  with  the  3D  and 
interacts  with  the  2D  on'iy  when  he  specifically 
asks  for  help. 

In  the  evaluation  stage,  the  2C  continues  to 
monitor  individual  actions  to  maintain  records  of 
student  performance.  The  interaction  monitored 
will  be  almost  entirely  with  the  3D,  and  vir¬ 
tually  no  video  or  computer  generated  messages 
will  be  used. 

THE  FUTURE  OF  SKIluS  TRAINING. 

ISTs  are  currently  being  evaluated  for  train¬ 
ing  effectiveness  and  transfer  of  learning.  Their 
potential  for  providing  effective  training  in  all 
three  learning  stages  will  be  realized  as  lessons 
are  developed  which  use  thy  diversity  of  methods 
and  techniques  available,  and  are  applied  across 
a  wiae  range  of  training  programs. 

New  applications  are  in  progress.  Tech¬ 
nology  for  training  devices  is  constantly  im¬ 
proving.  Microprocessor  technology  continues  to 
grow.  The  new  generation  of  videodisc 
development  will  1 5 ke ly  include  single  frame 
audio  and  programmable  videodiscs.  Integrated 
Systems  Trainers  are  the  future  of  skills  train¬ 
ing. 
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ABSTRACT 

'■‘The  task  of  rraintainlng  complex  military  hardware  In  the  field  has  become  a 
major  problem  facing  all  the  military  services.  Contributing  to  this  problem 
are  the  mass  of  highly  technical  and  I nstruct I ona I !y  Inadequate  manuals  and  the 
low  reading  ability  and  Inexperience  of  many  military  technicians.  Solutions 
have  been  attempted  In  several  areas,  mostly  at  high  cost  and  with  little 
success.  One  promising  solution  appears  to  be  the  use  of  computer-based 
job-aids  for  the  technician  In  the  field.  One  such  system  Is  NOMAD  (Navy 
On-board  Maintenance  Aiding  Device),  a  prototype  computer-based  Job-aid 
developed  by  Hazel  tine  Corporation's  TICCIT  division  and  U.S.  Navy  personnel 
for  ship-board  maintenance  of  tho  Navy's  MK-86  fire-control  system.  NOMAD  has 
proved  very  successful  In  Its  Initial  tryout. 


THE  PROBLEM 

One  of  the  most  important  tasks  facing  all  the 
military  services  today  Is  that  of  maintaining  com¬ 
plex  systems  used  in  the  field.  The  volume  of 
printed  documental i on  that  has  accompanied  the 
introduction  of  i.uch  systems  has  grown  so  large 
that  much  of  It  cannot  be  accessed  quickly  enough 
to  make  it  useful  for  field  level  maintenance.  In 
addition,  such  information  is  difficult  to  store, 
difficult  to  update,  and  often  written  and  Illus¬ 
trated  at  too  high  a  level  to  be  intelligible  to 
military  technicians.  The  problem  Is~  further 
aggravated  by  the  low  reading  ability  of  many  of 
today's  military  personnel. 

The  magnitude  of  fhe  military  maintenance  prob¬ 
lem  has  been  thoroughly  documented. ^ >2,3  jpo  crux 
of  the  problem  has  been  identified  as  the  Inability 
of  technicians  to  troubleshoot  and  repair  complex 
systems  rapidly  and  accurately.  This  problem  Is 
manifested  by  a  high  percentage  of  errors  In  troub¬ 
leshooting  and  extended  and  eventually  unacceptable 
time  to  repair  the  equipment. 


ATTEMPTED  SOLUTIONS 

Approaches  In  the  following  areas  have  been 
taken  to  attempt  to  overcome  the  military  mainte¬ 
nance  problem: 


Let  us  now  consider  each  of  these  approaches  In 
detal I . 

1.  Formal  and  on-the-job  training.  Despite  In¬ 
creased  expend  I tures  on  tral n! ng  In  the  military 
community,  this  approach  has  had  limited  success 
for  the  following  reasons: 

a.  Long-term  delays  between  training  and  on- 
the-job  applications.  A  particular  fault  may  occur 
so  infrequently  as  to  make  It  Impossible  for  the 
technician  to  be  practiced  at  locating  and  correct¬ 
ing  It.  Such  a  fault  may  occur  years  after  the 
pert  I nent  trai nl ng. 

b.  Sophistication  of  equipment.  The  task  of 
the  technician  becomes  more  sophisticated  when  mod¬ 
ularization  Is  Introduced  because  he  or  she  must 
deal  with  complex  inter-relationships  among  whole 
subsystems,  rather  than  individual  faulty  parts. 

c.  Out-dated  training  techniques.  With  only  a 
few  exceptions  (such  as  the  use  of  modern  computer- 
based  training  systems)  many  military  maintenance 
tral nlng  techniques  are  twenty  to  thirty  years 
behind  accepted  learning  theories  and  practices. 

d.  Loss  of  skills.  Usually,  forma  I ized  m  1 1 I- 
tary  training  programs  for  technicians  do  not  allow 
for  refresher  and  updated  training  while  on  the 
job.  Necessary  detailed  knowledge  and  skills  are, 
therefore,  lost. 


1.  Formal  and  on-the-job  training 

2.  Modularization  of  weapons  systems 

3.  Development  of  Built-in  Test  Equipment  (BITE) 

4.  Development  of  Automatic  Test  Equipment  (ATE) 

5.  Development  of  high  quality  technical  manuals 

6.  Research  on  the  fundamentals  of  the  man-machine 
1 nterf ace 


2.  Modularization  of  weapons  systems.  While  modu¬ 
larization  was  Intended  to  simplify  the  techni¬ 
cian's  task  by  having  him  or  her  remove  the  faulty 
module  and  replace  it,  the  success  of  this  approach 
depended  on  the  diagnostic  abll Ity  of  the  techni¬ 
cian,  the  ability  of  the  technician  to  replace  the 
module  without  damaging  the  system,  and  the  avail¬ 
ability  of  replacement  modules.  The  first  two  of 
these  factors  are  dependent  on  the  quality  of  the 
personnel  entering  the  training  system  and  the 
ability  of  armed  forces  to  retain  expert  techni¬ 
cians.  Both  of  these  areas  have  been  problems  in 
recent  years. 
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3.  Development  of  Built-in  Test  Equipment  (BITE). 
BITE  is  designed  into  the  system  to  be  tested  at 
the  time  of  Its  construction  and  therefore  must  be 
designed  at  the  same  time  as  the  system  itself. 
Unfor+unately,  It  is  usually  not  possible  to 
forecast  accurately  all  possible  types  of  faults  at 
the  time  of  system  design.  In  addition,  BITE  has 
proven  unsatisfactory  because: 

a.  Deficiencies  in  GiTE  are  extremely  diffi¬ 
cult  to  correct,  since  BITE  is  initially  de  igned 
into  the  system. 

b.  If  it  is  to  identify  more  tnar.  60?  of  tne 
faults  in  a  complex  system;,  BITE  must  be  so  complex 
itself  that  it  becomes  hard  to  troubleshoot  and 
mai ntai n. 

c.  BITE  does  not  provide  aids  for  performing 
replace  and  repair  functions. 

4.  Development  of  Automatic  Test  Equipment  (ATE). 
ATE  consists  of  very  large,  complex,  and  expensive 
devices  designed  to  be  connected  to  the  system  to 
be  tested.  ATE  is  commonly  used  for  troubleshoot¬ 
ing  and  maintenance  of  military  aircraft  such  as 
the  F— 1 6 .  ATE  is  Impractical  in  many  maintenance 
and  troubleshooting  situations  because  of  its  ex¬ 
pense  and  size.  This  approach  is  only  feasible  for 
depot  level  maintenance. 

5.  Development  of  high  quality  technical  manuals. 
Such  an  approach  has  not  proved  effective  because: 

a.  Paper  manuals  are  bulky  and  hard  to  handle. 

b.  Paper  manuals  are  difficult  to  updo  re. 

c.  Access  time  is  excessive  when  large  numbers 
o4  manuals  are  i.eeded  to  explain  a  single  system. 

d.  Information  in  this  form  is  usually  narC  +o 
understand  and  is  usually  not  designed  to  meet  the 
needs  of  different  levels  of  users  4rom  novi.es  to 
experts. 

6.  Research  on  the  fundamentals  of  the  man-machine 
I  nr erf  ace.  Ail  the  approaches  described  above  have 
proved  deficient.  Therefore,  further  research  into 
the  man-machine  interface  appeared  to  be  necessary. 
Several  directions  for  research  have  been  followed; 

a.  One  proposed  research  so  I  ut  I  c-r  to  the  prob¬ 
lem.  has  been  to  study  the  manner  in  which  expert 
technicians  trout)! eshoot  and  maintain  complex  sys¬ 
tems.  It  was  hoped  that  an  increased  understanding 
of  expert  knowledge  would  lead  to  improvements  I r. 
training  techniques  for  novice  technicians.  Al¬ 
though  much  interesting  work  has  been  done  In  this 
are;,  no  major  i  provements  ha' e  yet  taken  place  in 
the  training  of  technicians,  and  as  a  rosult  no 
major  Improvements  In  weapons  system  availability 
have  resulled  from  this  research. 

b.  A  second  potential  soluv'on  Is  to  model 
complex  systems,  such  as  steam  ►  ants,  in  compu¬ 
ters.  It  is  thought  that  an  Improved  understanding 
cf  how  such  systems  function  and  how  pec*  <©  oper¬ 
ate,  troubl  ©sheet,  and  maintain  such  systems  till 
result  In  Improved  approaches  to  trouble: ryy* I  ng 
and  maintenance.  Such  simulations  are  b.jsi.-c  on 


matnematical  models  of  complex  Systems.  The 
meters,  dials,  and  other  indicators  of  system  func¬ 
tioning  are  represented  as  graphics  on  computer 
displays.  Mile  this  approach  may  have  value  in 
the  distant  future,  it  seems  clear  that  ro  major 
improvements  in  training  or  maintenance  and  trou¬ 
bleshooting  may  be  expected  to  develop  In  tie  near 
future.  These  complex  simulations  are  based  on 
mathematical  models  that  are  extremely  expensive  to 
develop,  and  as  yet  they  have  had  little  inpact  on 
weapons  system  availability. 

c.  A  third  approach  is  to  employ  computer¬ 
ised  job-aids  designed  to  deliver  troubleshooting 
and  maintenance  Information  in  the  field,  in  this 
approach  a  computer  terminal  with  access  to  an  in¬ 
teractive  dala  ba»e  is  made  available  to  the  tech¬ 
nician  at  the  job  site.  An  example  of  this  is  Ihe 
U.S.  Navy  sponsored  effort  to  deveiop  a  prototype 
of  one  such  system  for  ship-board  use.  This 
device,  called  NOMAD  (Navy  On-board  Maintenance 
Aiding  Device),  provides  a  structured,  aufomated 
diagnostic  strategy  that  prompts  and  logical iy 
leads  technicians  through  appropriate  procedures 
end  actions  in  troubleshooting  the  Navy's  MK-66 
fire-control  system.  The  ^K-86  fire-control  :ystem 
is  an  extremely  complex  gunfire  control  iystem 
designed  to  protect  Spruance  class  destroyer:  from 
hostile  aircraft  and  missiles.  Although  the  W<-86 
system  is  effective  when  operating  properly,  it  has 
been  prone  to  failure  due  to  a  shortage  of  the 
experienced  technicians  required  to  troubl eshoot 
and  maintain  i  h.  The  remainder  o4  this  paper  des¬ 
cribes  the  application  of  NOMAD  to  this  problem. 


NOMAD :  A  PROTOTYPE  SOLUTION 

NOMAD  is  currently  aboard  the  U.S.S.  Kinkeid, 
a  Spruance  class  des+royer  in  the  Pacific,  where  It 
Is  undergoing  test  and  evaluation.  NOMAD  is  be'ng 
used  or  the  Kinkald  for  Mi' -86  system  familiarisa¬ 
tion  (for  junior  technicians),  troubleshooting  and 
diagnostic  procedure  prompting,  fault  isolation 
prompting,  W-86  checkout  and  grooming,  and  presen¬ 
tation  of  videotape  malrtal nance  procedure  sequer- 
ces.  (NOMAD  is  capable  of  both  videotape  and  vide¬ 
odisc  presentations  integrated  I  r.to  computer-gener 
a ted  text  and  graphics,  under  computer  control.) 

The  scope  of  the  troubleshooting  and  mainte¬ 
nance  materiel  currently  available  on  NOMAD  Is 
I  imlted  to  these  areas  of  the  W-66  system  most 
prone  to  failure.  This  subset  of  material  was 
determined  by  in*.-  Navy  to  be  adequate  for  purposes 
of  this  evaluation. 

Physically  the  NOMAD  system  is  composed  of  a 
minicomputer  and  peripheral  hardware  located  on¬ 
board  In  the  ship's  data  processing  center.  Two 
Interactive  color  lerm Inals  with  keyboards  and 
light  pens  are  strategically  placed  within  the  ship 
to  a  I  low  ready  access,  to  the  **-86  hardware.  One 
terminal  is  located  In  Radar  Room  Number  1  and  the 
other  in  tne  Combat  Information  Center.  The  cur¬ 
rent  two  terminal  system  Is  expandable  to  32  termi¬ 
nals  should  the  need  arise.  The  repair  technician 
communicates  through  the  NOMAD  terminal  with  The 
troubleshooting  da4aoa:.e  and  a'gorlthm  stored  in 
the  computer . 
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PROGRAM  DESIGN  CONSIDERATIONS 

Key  aspects  «.>(  Ine  NOMAD  progrtui  mciuut  me 
use  of  (a)  commercially  available,  off-the-shelf 
hardware  and  software,  only  s I ! ght I y,  mod  I f led  for  a 
shipboard  environment  (NOMAD  is  a  compact  version 
of  the  TICCIT*  computer-assisted  Instruction  system 
manufactured  by  Hazel  tine  Corporation),  (b)  U.S. 
Navy  enlisted  personnel  (one  Senior  Chief  and  two 
sailors/  to  provide  The  subject  matter  expertise, 
create  the  troubleshooting  and  maintenance  mate¬ 
rial,  and  put  it  on  the  computer,  and  (c)  rapid 
implementation  of  the  concept  (seven  months  from 
cona  pt  to  functioning  hardware  and  software  aboard 
the  n i nka i d . 


The  significance  of  NOMAD  as  a  prototype  device 
lies  not  in  Its  hardware  but  rather  in  the  toubie- 
shooting  algorithm,  database,  and  presentation  mode 
employed  in  the  system. 

The  troubleshooting  strategy  implemented  in 
NOMAD  adapts  to  the  level  of  expertise  of  the  user, 
leads  him  or  her  to  the  fault  in  the  system,  and 
prescribes  the  required  correct  ve  action.  Where 
possible,  the  system  references  the  appropriate 
technical  documentati cn  rather  than  duplicating  It, 
NOMAD  also  keeps  detailed  records  of  each  techni¬ 
cian's  use  of  the  system  during  troubleshooting. 
The  major  goal  in  implementing  such  a  system  is  to 
deliver  the  expert's  knowledge  to  the  technician. 
This  must  be  accomplished  by  first  transferring 
that  knowledge  to  the  computer.  Ideally  this 
transfer  should  be  made  rapidly  and  inexpensively, 
which  requires  an  easy-to-use  software  system  on 
which  the  subject  matter  expert  (SME)  may  work 
directly  to  ’ mpart  h!s  or  her  knowledge  and  with 
which  the  end  user  can  interface  easily  and  effec¬ 
tive  I  y . 

These  considerations  inaicate  a  need  to  focus 
on  the  man-computer  interface  rather  than  the  com¬ 
puter  hardware  package.  There  is  a  great  tempta¬ 
tion  for  those  working  in  this  area  to  attempt  a 
hardware  solution  to  this  problem— that  is,  to  de¬ 
cide  in  advance  that  a  computer-based  job-aid 
should  be  of  a  certain  weight,  of  a  certain  size, 
have  a  display  area  of  certain  parameters,  etc. 
Rather  than  take  this  approach.  Hazel  tine  Corpcra- 
tion  staff  are  systematically  focusing  on  what  we 
ccnsidar  to  be  the  crucial  concerns  here,  all  of 
which  are  aspects  of  the  man-computer  interface. 
Specifically,  a  computer-based  job-aid  shculc: 


1 .  Make  the  knowledge  of  the  SHE,  available  to  The 
novice  in  a  form,  -"hat  .makes  it  easy  to  understand 
and  effective  In  helping  him  or  her _ s.0.1..ve..  rails  ter 

gram  is  based  on  the  idea  that  It  is  not  necessary 
to  provide  a  deep  structure  representation  of  the 
SME's  knowledge  In  the  computer-based  job-aid.  All 
that  is  required  is  that  the  expert  make  explicit 
his  or  her  troubleshooting  and  maintenance  approach 
and  then  represent  that  approach  in  the  computer's 
programs i ng.  Neither  is  it  necessary  to  attempt  to 
arrive  at  some  objective  specification  of  exper¬ 
tise.  Instead,  a  single,  effective  approach  Is 
made  available  to  the  i nexper i enced  but  school- 
trained  technician.  Experts  differ  ir  their  con¬ 
ceptualizations  of  complex  systems  and  In  their 
strategies  in  troubl  eshc-oti  ng  such  systems.  A I  i 
that  is  really  needed,  however,  is  one  approach 
that  works. 
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? .  Utilize  a  user-friendly  software  system  tp  cre- 


oedurcs.  MOMAD  ut!  i  !zcc  or,  existing  commercial 
user-friendly  language,  called  TAL  (TICCIT  Author¬ 
ing  Language),  which  was  designed  for  the  authoring 
of  flexible  Instruction?!  presentations  and  simula¬ 
tions.  It  has  also  pi  oven  effective  and  easy  to 
use  •  +  t_e  present  application.  Although  most  of 
the  authoring  *as  performed  at  a  land-based  facil¬ 
ity  before  Installation  of  the  on-board  system,  the 
system  allows  for  on-  ' te  authoring  as  necessary. 


3 .  Ulilizfi-jB  jjser.-frlenciy  graphics  systmi  tO-QEfir 
ate  graphics  of  wave  forms.  Instrument  panel s a... etc. 
A  commercially  available  document  camera,  part  of 
the  TICCIT  system,  was  used  for  putting  NOMAD 
graphics  on  line.  This  camera  and  its  associated 
software  package  allow  graphics  drawn  by  illustra¬ 
tors  or  found  in  Navy  technical  manuals  to  be  scan¬ 
ned  into  the  computer  system  in  minutes.  These 
graphics  are  then  edited  using  an  on-line  graphics 
software  package.  Graphics  are  colored,  rotated, 
expanded,  and  reduced  as  required.  The  graphics 
editor  can  also  be  used  to  create  unique  symbols  or 
characters,  which  can  be  assigned  to  keys  on  the 
keyboard  and  stored  for  subsequent  use.  The  ease 
with  which  one  can  construct  graphics  makes  it  pos¬ 
sible  to  staff  an  effort  like  the  NOMAD  program 
with  inexperienced  personnel,  such  as  entry- level 
enlisted  people.  The  document  camera  is  not  part 
of  the  ship-boaro  hardware  package,  so  graphics 
were  digitized  at  the  iand-based  facility.  However, 
the  graphics  software  package  is  available  ship¬ 
board  to  make  any  necessary  mod i i icati ons  identi¬ 
fied  on  site. 


NOMAD  graphics  are  displayed  instantaneously  as 
one  imago  rather  than  being  drawn  slowly  as  the 
technician  watches.  In  situations  that  require  the 
use  of  complex  graphics,  this  high-speed  display 
capability  is  criticai  to  clarifying  complex  tech¬ 
nical  text. 


4 .  Utilize  state-of-the-art  instructional  design 
techniques  such  as  hy pertexf  to  ensure  that  t 1 , 


possible.  Data  are  presented  at  various  levels 
from  direct  access  for  the  experienced  technician 
to  a  guided  step-by-step  approach  for  the  novice, 
with  a  wide  variety  of  possiblities  between.  The 
mjjo:  interface  between  the  technician  and  the 
computer  is  through  a  light  pen  for  pointing  to 
locations  on  the  terminal  screen  rather  than  by 
typing  inputs  on  the  terminal  keyboard.  This 
speeds  up  the  troubleshooting  process  and  avoids 
the  prubiur  of  pour  spelling  and/or  typing  skills 
on  the  technician's  part. 


PRELIMINARY  RESULTS 

At  the  present  time,  very  little  quantitative 
data  on  the  preliminary  try-out  has  been  made 
available.  However,  the  existing  results  indicate 
that  this  approach  is  very  successful  in  increasing 
weapons  system  availability.  We  expect  to  receive 
enough  accitional  data  about  NOMAD's  performance  to 
draw  meaningful  conclusions  about  the  following 
topics: 

1.  Reliability  of  fte  ruggedized  commercial  hard¬ 
ware  in  the  shipboard  environment. 


- 
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2.  Attitudes  of  U.S.  Navy  technicians  toward  using 
an  I nteractl ve  computer-based  Information  system. 

3.  An  assessment  of  the  potential  of  the  system  to 
significantly  Improve  W<-86  operational  readiness 
and  sys'yn  availability  indices,  g'ven  an  expanded 
data  base  to  cover  all  suitable  MK-86  subsystems. 
(Only  a  subset  of  such  systems  are  now  Included  i n 
the  NOMAD  programm  ing. ) 

Tentative  results  in  these  areas  are  summarized 
here: 

1.  Reliability.  The  prototype  NOMAD  system  is  not 
a  rugged i zed  system  except  for  six  shock  mounts 
attached  to  the  equipment  rack.  Nevertheless,  the 
off-the-shelf  system  hardware  has  survived  at  sea 
since  January,  1982,  and  has  continuously  func¬ 
tioned  well  with  only  short,  minor  i nterrupti ons 
such  as  were  expected  in  this  first  test  of  the 
system  at  sea.  After  an  initial  break-in  period, 
the  Navy  technicians  have  been  successful  in  opera¬ 
ting  NOMAD  at  sea  with  little  or  no  help  from  con¬ 
tractor  representai  es,  despite  having  received 
only  cursory  hardware  training.  Successful  opera¬ 
tion  has  been  performed  during  the  firing  of  the 
ship's  5- inch  guns  and  at  times  of  heavy  seas. 

We  can  only  assume  that  a  fully  rugqedized 
production  version  of  the  system  would  perform  as 
wel I  as  or  better  than  the  present  system. 

2.  Technicians’  attitudes  toward  such  a  system. 
Technicians’  attitudes  have  been  very  positive 
toward  the  system.  In  fact,  they  would  like  to  see 
additioric1  c-ogramming  to  cover  other  sections  of 
the  M<-86.  Technicians  were  also  Impressed  with 
the  ability  of  the  sy to  present  videotapes  of 
mainfenance  procedures,.  Favors '<?  reactions  have 
been  expressed  by  crew  members  of  othe-  ships  who 
have  seen  the  system  demonstrated,  as  wel I  as  by 
various  visiting  dignitaries.  Attitudes  towards 
the  NOMAD  system  were  so  positive  that  the  Captain 
of  the  Kinkald  requested  that  the  system  be  retain¬ 
ed  on  the  ship  rather  than  be  removed  at  the  end  of 
six  months  as  had  been  original ly  planned.4  This 
requst  was  granted.-’ 

3.  System  potential.  The  following  data  on  sample 
fault  Isolation  times  w ■ fh  and  without  NOMAD  have 
been  provided  by  Naval  Ship  Weapon  Systems  Engi¬ 
neering  Station  (NSWSES),  Port  Hueneme,  California, 
for  the  photo  tube  assembly  (10A5A5)  ana  power  sup¬ 
ply  (19PS5).6  Data  without  NOMAD  come  from  other 
ships  as  well  as  the  Kinkald.  Individual  times 
presented  s  ^present  separate  events. 


Troubleshooting  Time  (hours) 


MK-86 

Subsystem 

with 

NOMAD 

without 

NOMAD 

Photo  Tube 

.50 

1 .60 

Assembly 

.17 

3.20 

1 .40 

.20 

Average 

M60 

Power  Supply 

.33 

.21 

.50 

.50 

.90 


Average _ .53 


it  has  been  recommended  that  "NOMAD  should  be 
expanded  to  encompass  the  entire  VK-86  system  to 
truly  prove  its  value  as  [a]  troubleshooting  and 
maintenance  aid  device."^ 

Ideally,  there  would  have  been  a  formal  effort 
to  collect  quantitative  data  for  the  above  cate¬ 
gories  of  information  and  compare  them  to  baseline 
data.  This  was  not  possible,  so  instead  a  general 
assessment  is  being  collected  for  each  category 
based  on  at-sea  experience  as  reflected  in  appro¬ 
priate  system  logs.  Additional  data  will  come  from 
narrative  reports  from  chiefs  and  officers  in  the 
weapons  department. 

The  reports  prepared  on  NOMAD  include: 

1.  A  biweekly  Situation  Report  from  the  USS  Kin- 
kaid,  which  includes  comments  on  the  usefulness  of 
iJOMAD  as  a  maintenance  tool  and  as  a  tutorial  aid. 

2.  A  monthly  report  from  NSWSES  containing  an 
araiysis  of  all  data  co Me  ted  in  the  month 
reported. 

3.  A  prel iminary  report  from  NSWSES,  scheduled  for 
published  in  October,  1982,  after  completion  of  the 
trial  period,  addressing  the  analysis  results  for 
Navai  Sea  Systems  Command. 

4.  A  final  report  from  NSWSES,  scheduled  for  pub¬ 
lication  in  November,  1982,  for  Naval  Sea  Systems 
Commano  approval  and  distribution. 

THE  FUTURE 

It  should  be  noted  that  in  addition  to  its 
NK-86  related  functions,  current  additional  uses 
for  NOMAD  aboard  the  Kinkald  include  shipboard 
indoctrination,  EPICS  (Enlisted  Personnel  Indivi¬ 
dualized  Career  System)  training  support,  and 
recreational  applications.  NOMAD  could  also 
deliver  the  same  instructional  materials  available 
on  any  of  the  TICCIT  systems  currently  in  use  for 
Navy  and  Marine  Corps  training. 

The  success  of  NOMAD  has  cleared  the  way  for 
future,  enlarged  versions  of  this  prototype  system. 
An  RFP  has  been  received  by  Hazel  tine  concerning  a 
second  system  for  use  on  an  Atlantic-based  ship. 
Development  of  a  more  compact  version  of  NOMAD 
hardware,  based  on  a  microcomputer.  Is  now  under 
way. 

MjCh  of  what  has  been  learned  about  how  to 
design  job-aMlng  material  during  the  NOMAD  program 
is  applicable  to  other  military  hardware  systems. 
The  potential  for  this  solution  to  providing  help 
to  technicians  in  the  field  seem  unlimited. 
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LESSONS  LEARNED 
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ABSTRACT 

Technical  training  and  the  isolation  and  diagnosis  of  jet  engine 
malfunctions  has  traditionally  been  accomplished  using  operational  engine 
hardware,  which  has  limited  malfunction  training.  Simulated  aircraft 
maintenance  training  (SAMT)  devices  are  beirg  increasingly  employed  by  the 
military  to  achieve  more  efficient  and  controlled  instruction  in  maintenance 
procedures.  The  F-16  engine  diagnostic  SAMT  is  comprised  of  simulated  aircraft 
cockpit  and  test  equipment  control  panels,  an  instructor  station,  and  a  computer 
simulation  of  the  Pratt  &  Whitney  F-100  engine.  The  math  model,  which  consists 
of  a  data  base  of  engine  variables,  with  transients  provided  by  simple 
algorithms,  was  found  to  provide  completely  realistic  engine  performance  for 
maintenance  training.  Through  the  model,  students  can  practice  trimming 
procedures,  and  diagnosis  of  a  variety  of  engine  component  failures.  Valuable 
lessons  were  learned  in  regards  to  sources  of  data  for  data  base  and  algorithm 
development,  data  base  fidelity,  and  approaches  to  malfunction  model 
development.. 

\ 


BACKGROUND 

Technical  training  in  jet  aircraft 
engine  trimming  procedures  and  m  the 
isolation  and  diagnosis  of  jet  engine 
malfunctions  has  traditionally  been 
accomplished  by  using  operational  engine 
hardware;  either  a  complete  aircraft  or 
the  engine  assembly. 

The  use  of  "Hot  Mockups"  for  teaching 
engine  trimming  has  the  drawbacks  of  (1)  a 
noisy  teaching  environment,  (2)  exposing 
the  student  to  hazards  which  are  not  a 
vital  part  of  learning  how  to  trim  an 
engine,  (3)  the  always-present  competition 
with  other  groups  for  aircraft  hardware; 
historically,  training  usually  has  a  lower 
priority  than  operations  and  maintenance, 
and  (4)  trimming  an  engine  burns  a 
substantial  amount  of  fuel  and  results  in 
significant  wear  on  the  engine.  Data  from 
recent  USAF  engine  trim  courses  indicate 
that  it  requires  30,000-40,000  lbs  of  fuel 
and  approximately  8  hours  of  engine  time 
to  train  each  student. 

For  these  reasons,  simulated  aircraft 
maintenance  training  devices  are  being 
increasingly  employed  by  the  military  to 
reduce  costs  and  achieve  more  efficient 
and  controlled  instruction  in  maintenance 
procedures . 


THE  F-16  ENGINE  DIAGNOSTICS  SAMT 

An  example  of  this  new  approach  to 
maintenance  training  is  the  F-16  Engine 
Diagnostic  Simulated  Aircraft  Maintenance 
Trainer  (SAMT),  comprised  of  two  3’  x  8' 
panels,  an  instructor  station  and  a 
computer  simulation  of  the  Pratt  and 
Whitney  F-100  engine.  The  two  panels 
contain  aircraft  cockpit  instruments, 
engine  drawings  and  blowups  of  selected 
components  as  well  as  relevant  test 
equipment  control  panels.  They  also 
contain  a  set  of  130  action  switches, 
which  allow  the  student  to  simulate  the 
taKing  of  various  actions;  for  example,  an 
action  switch  is  used  for  applying  ground 
power.  A  set  of  70  element  switches 
are  distributed  within  the  panels  which 
allow  the  student  to  designate  particular 
components;  for  example,  an  element  switch 
is  used  to  designate  the  engine  alternator 
as  a  defective  component. 

The  instructor's  station  is  comprised 
of  a  CRT  display  and  keyboard  which  allows 
the  instructor  to  choose  one  of  a  set  of 
engine  malfunctions  for  the  current  lesson 
and  to  record  student  information  relevant 
to  the  training.  An  instructional  feature 
of  the  software  prompts  the  student  with 
caution  and  hazard  messages,  and  records 
student  performance  as  the  lesson 
progresses . 
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Tr.e  engine  model  and  test  set  models 
interact  to  form  a  totally  free-play 
environment.  The  student,  is  free  to  make 
mistakes  on  both  engine  operation  and  test 
-et  operation,  and  the  system  is  designed 
to  give  the  appropriate  response.  A 
simple  real  time  monitor  is  included  in 
the  simulation  to  detect  and  flag  student 
operational  and  procedural  errors. 

Traditional  approaches  to  jet  engine 
modeling  have  involved  mathematical 
description  of  complex  mechanical  and 
thermodynamic  processes.  This  approach 
when  applied  to  maintenance  trainers  is 
both  costly  and  unnecessary. 


THE  ENGINE  MODEL 

A  new  approach  was  chosen  for  the 
[..•nth  model  of  the  engine  which  is  driven 
by  a  data  base  of  engine  variables  that 
describe  the  steady-state  behavior  of  the 
engine.  The  transient  responses  are 
provided  by  simple  algorithms.  This 
approach  was  found  to  provide  completely 
realistic  engine  performance  for 
maintenance  training. 


The  engine  data  base  consists  of  22 
tables  which  describe  the  engine 
variables  in  normal  and  diagnostic  modes. 
The  normal  engine  parameters  data  consist 
of  tabulated  values  for  nine  observable 
parameters;  fuel  flow,  nozzle  position, 
variable  vane  position,  compressor  and 
turbine  RPM,  two  temperatu res ,  and  two 
pressures.  In  addition  to  the  normal 
engine  tables,  data  is  tabulated  for  the 
engine  operatinng  without  the  engine 
electronic  controller  (EEC)  and  with  the 
back-up  controller  (BUC).  Seven  different 
engine  trim  tables  and  12  malfunction 
tables  complete  the  3et, 

Each  of  the  tables  (Refer  to  Table  1) 
is  composed  of  three  sub-tables;  one  for 
each  of  three  values  of  outside  air 


temperature .  For  a  given  outside  air 
temperature,  the  steady-state  value  of 
each  of  the  nine  parameters  is  tabulated 
.or  incremental  values  of  throttle 
setting.  The  tabulated  values  were 
obtained  using  the  manufacturer 1  * 
comprehensive  non-rea 1- t ime  engine 
simulation  program. 

Refinements  to  the  tabulated  values 
are  then  made  to  the  nine  parameters  to 
simulate  the  function  of  six  trim  screws, 
engine  electronic  controller,  back-up 
controller,  ambient  air  temperature,  air 
source  selector,  anti-ice  switch,  starting 
fuel  switch  (lean/rich),  false  parameters 
introduced  by  various  engine  test  sets, 
and  parameter  perturbations  due  to 
simulated  malfunctions.  (Refer  to  Figures 


The  model  includes  3 2  classifications 
of  malfunctions  which  fault  isolate  to  75 
unique  problems.  The  malfunction  models 
are  either  table-driven,  algorithm-driven 
or  are  a  combination  of  the  two. 
Referring  to  Figure  1,  the  malfunction 
symptoms  are  either  inserted  during 
throttle  movement,  incorporated  into  the 
engine  controller  algorithms  (BUC  or  EEC), 
absorbed  in  the  steady-state  tables  or 
added  at  the  time  of  engine  transient 
response.  Figure  2  is  iTTcluded  to  show 
what  is  involed  in  calculating  a  typical 
parameter  in  real  time. 


Engine  transients  involve  the  time 
behavior  of  parameters  toward  the 
steady-state  values  corresponding  to 
tnrottle  position.  A  satisfactory  and 
simple  method  which  we  use  to  model  these 
transients  proved  to  be  an  exponential 
response  for  each  engine  parameter  of  the 
form.  (Refer  to  Figure  3.) 


Where 

A  =  Steady-state  value  of  the  parameter  at 
initial  throttle  setting  A. 

B  =  Steady-state  value  of  the  parameter  at 
final  throttle  setting  B. 

X(T)  =  Value  of  the  parameter  at  time  T, 

TC  =  Time  constant 

Note  that  the  classical  form  of  the 
response  in  eq  (1)  is  computationally 
difficult  and  involves  the  evaluation  of 
exponentials.  A  more  efficient  form  is 
found  by  developing  a  recursion 
relationship: 

From  (1)  letting  t  =  NAT 
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TABLE  1 


TYPICAL  DATA  BASE  TABLE 


Throttle 

Ambient  Temperature 

Parameter 

( Degree ) 

C  deg  45  deg  ICO  deg 

RPM 

20 

- 

- 

- 

23 

- 

- 

- 

26 

- 

- 

- 

3  c 

- 

- 

- 

44 

- 

- 

- 

50 

- 

- 

- 

68 

- 

- 

-- 

80 

- 

- 

- 

83 

- 

- 

- 

NOZZLE 

20 

- 

- 

- 

23 

- 

- 

- 

38 

- 

- 

- 

50 

- 

- 

- 

53 

- 

- 

- 

6  5 

- 

- 

- 

80 

- 

- 

- 

91 

- 

- 

- 

109 

- 

- 

- 

1 15 

- 

- 

- 

127 

- 

- 

- 

130 

- 

- 

- 

TEMPERATURE  20 

23 
26 
32 

3  c 

42 

45 

51 

5° 

68 

79 

83 


(etc  for  all  10  Parameters) 
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FIGURE  2  ENGINE  DIAGNOSTICS  SIMULATION 
(CALCULATION  OF  A  TYPICAL  PARAMETER) 
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_  NAT  _NAI 

X ( N )  -  Ae  TC  +  B  [1  -  e  TC] 

by  substitution  from  (1)  and  collecting 
terms 

_  _AT 

X(N+1)  =  e  TC  X(N)  +  B(1  -  e  TC  )  ^ 


In  other  words,  for  each  time  step, 
the  current  value  is  determined  by  adding 
K  times  the  difference  between  the  forcing 
function,  IN(N+1),  and  the  last  parameter 
value,  OUT ( N ) ,  to  the  last  parameter 
value  (Refer  to  Figure  3).  If  the  forcing 
function  (throttle  position)  remains 
constant,  this  is  a  classical  exponential 
response  to  a  stimulus. 


LESSONS  LEARNED 


Equation  (2)  can  be  written  in  a  n.or° 
computationally  efficient  form: 


'Tings  We  Did  W rone 


OUT ( N+  7 )  =  OUT(N)  +  [IN(N+1)  -  0UT(N)]*K 

Where 

0UT(N+1)  =  Current  value  of  the 

parameter 

OUT(N)  =  Value  of  the  parameter 

computed  at  the  last  AT  time  step 

I N ( N  + 1 )  =  Current  steady  state  value 
parameter  (a  function  of  throttle  setting) 

K  =  K  (AT ,  TC ) 

TC  =  Time  constant 

AT  =  Calculating  increment  (0.1 

second  in  our  case) 


Improved  Malfunction  Definition . 
Ir  the  early  stages  of  trainer 
development,  not  enough  attention  was  paid 
to  malfunction  definition.  Since  our 
model  does  not  directly  employ  the  basic 
physical  principles  of  the  engine,  but 
rather  relies  on  a  data  base  for 
symptomatic  description,  the  secondary 
malfunction  responses  are  not  inherent  in 
the  model.  (An  example  of  a  primary 
response  is  an  oscillating  exhaust 
nozzle,  whereas  the  secondary  responses 
are  the  changes  to  the  other  engine 
parameters  which  occur  as  a  consequence  of 
the  fluctuating  nozzle.)  The  modeling  of 
these  minor  (but  important  to  diagnostic 
training)  responses  must  be  carefully 
specified.  However,  because  of  their 
secondary  nature,  it  is  difficult  to 
obtain  general  agreement  on  what  these 
responses  should  be.  The  manufacturer's 
data  was  generally  accepted  as  the  most 
reliable  and  tended  to  resolve  the 
differences.  In  retrospect,  the 

manufacturer  should  have  been  consulted  in 
more  detail.  We  also  we  should  have 
relied  more  heavily  on  F-15  engine  data, 
since  the  two  aircraft  have  the  same 
engine  and  F-15  had  been  operational  for 
some  time. 


Acded  Aud; 
provisions  w< 


2 _ Y-LSUs 

made 


1  Crt  les  ?  N  c 
for  special 


diagnostic  audio  responses  and  we 
discovered  at  a  rather  late  date  that  the 


engine  Job  Guides  require  the  maintenance 
man  to  listen  for  igniter  plug  noises.  He 


is  also  required  to  look  for  fire  coming 
from  the  tips  of  the  igniter  plugs,  end 
visual  indications  of  igniter  firing  were 
not  provided  for.  Adding  these  special 
audio  and  visual  effects  might  be 


desirable. 


0UT(N*1)  -  OUT ( N )  -  [1N(N*1)  -  0UT{N)J 


Things  We  Did  Riant 


Exponential  response  proved  to  be 
Simple  and  Effective.  The  selection  of 
this  type  of  response  to  describe  engine 


transient  phenomena  proved  to  be  a  wise 
choice.  We  employed  more  complicated 
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approaches  (notably  series  and  parallel 
networks  of  exponential  responses)  in 
other  trainers.  Where  these  approaches 
were  tried  on  the  Engine  Diagnostics  3AMT 
the  improvements  were  negligible  even  when 
instrumentation  was  employed. 


Improved  Method  of  Tracking  Equipment 
Changes .  As  the  engine  and  test  sets 
evolve  and.  new  operational  hardware 
appears,  the  problem  is  ever-present  on 
how  often  to  retrofit  the  trainers  and 


what  to  do  to  accommodate  the  missing 
hardware  in  the  meantime.  A  very  workable 
and  inexpensive  method  was  devised 
consisting  of  software  "bridges",  which 


are  a  CRT  message  to  the  student  to 
describe  equipment  responses.  For 
example,  if  a  voltage  is  to  be  measured 


and  a  voltmeter  is  not  included  on  t-  - 


trainer,  the  message  "Voltmeter  reads  22 
Volts"  appears  at  the  proper  time  in  the 


lesson . 


Model  Accommodates  Other  Jet  Engines . 
Since  transient  responses  are  not  of 
primary  importance  in  the  teaching  of 
engine  diagnosis,  (readings  are  always 
taken  after  the  engine  stabilizes)  the  use 
of  the  exponential  response  is  probably 
adequate  for  any  jet  engine.  It  is 
therefore  likely  that  the  model  car 
accommodate  most  any  jet  engine  by 
changing  the  data  base  (and  using 
appropriate  engine  parameters)  and  some 
time  constants. 

Things  We  Should  Have  Done  Better 


Reduced  Computation  Time.  Since  the 
engine  model  as  well  as  the  test  set 
models  are  table-driven,  the  basic 
trade-off's  in  table  structure  are: 


(1)  Polynomial  fits  to  the  data  with  no 
breakpoints  -  the  table  consists  of 
coefficients  of  the  polynomials. 


(2)  Breakpoint  logic  -  only  the  vertices 
are  retained  and  the  parameters  are 
computed  by  linear  interpolation.  (Refer 
to  Table  1 ) 


(3)  Slcpe/Intercept  form  -  the  vertices 
are  retained  as  in  breakpoint  logic  but 
the  slope  and  intercept  are  pre-corn}. uted 
and  stored  as  table  entries. 


Method  (1)  was  ruled  out  because  we 
were  unsure  of  the  fidelity  r-'c-ssary  fcr 
training  ard  high  fidelity  requires  a  very 
t igh  degree  of  polynomial  to  adequately 
describe  the  data.  Since  a  {oiynomiai  cf 
degree  N  requi res  N  multiplications  and  K 
additions  for  each  evaluation,  high  degree 
polynomials  are  precluded  because  of  ren. 
time  constraints. 


Method  (2)  requires  about  three  time: 
the  computational  time  of  that  cf  method 
>3  iri  order  tc  compute  a  parameter  value 
tut  requires  only  half  the  table  size, 
however,  Method  *2  was  choser  because  at 


the  time  cf  model  design  (1978  time  frame) 
computer  memory  was  considerably  more 
expensive  than  it  is  today. 

As  it  turned  out,  some  of  the  actual 
data  differed  significantly  from  the 
breakpoint  data  employed  and  proved  to  be 
entirely  satisfactory.  Since  the  actual 
breakpoint  data  provided  realistic 
performance,  we  now  think  that  we  could 
achieve  a  significant  improvement  by 
employing  various  data  compression  methods 
and  still  not  degrade  training 
effectiveness . 

Reduced _ Memory _ Requirements .  A 

similar  trade-off  exists  in  the  data  base 
structure.  The  current  method  of 
employing  different  engine  parameter 
tables  for  various  cockpit  switch 
combinations  and  malfunctions,  leads  tc 
large  memory  requirements,  which  could  be 
sharply  reduced  by  restructuring  the  data 
base  tc  include  one  primary  table  of 
engine  parameters  and  judiciously  adding 
switch-dependent  polynomial  fits  of  deltas 
to  these  basic  parameters.  In  fact,  we 
suspect  that  the  one  remaining  table  could 
be  significantly  reduced  by  opting  fcr  a 
coarser  data  structure  as  noted  above. 


MissinR _ Cockpit? .  All  engine 

maintenance  procedures  are  a  two  man 
operation:  One  man  is  in  the  cockpit  and 
the  other  is  operating  the  test  equipment. 
One  of  the  difficulties  facing  recert 
engine  training  class  graduates  is  how  to 
locate  the  many  cockpit  switches  that  must 
be  placed  in  the  OFF  or  SAFE  position  in 
orripp  to  make  the  aircraft  safe  for 
maintenance.  (This  procedure  must  preceed 
any  maintenance  action.)  It  may  be 
advantageous  to  include  a  simulated 
cockpit  with  many  of  the  switches 
dead-ended  but  moveable.  All  engire 
related  switches  would  be  operational. 

SUMMARY 

The  simple  model  consisting  cf  a  data 
base  of  steady-state  engire  parameters, 
coupled  with  ar.  exponential  response  for 
engine  transients  proved  to  be  a  very 
effective  approach. 

We  feel  now  that  we  could  go  back  and 
make  substantial  improvements  m  table 
structure  (data  resolution  and  data  base 
compression)  and  commutation  time. 

The  manufacturer  should  be  consulted 
r.oro  at  the  start  cf  the  program , 
particularly  in  the  area  of  malfunction 
definition.  In  doing  so  we  could  have 
avoided  the  problems  of  partially-  defined 
ard  ill-defined  malfunctions. 


262 


ABOUT  THE  AUTHORS 


Mr.  Thomas  J.  Stillings  is  a  Senior 
Principal  Systems  Analyst  Engineer  with 
Honeywell  Training  and  Control  Systems 
Operations.  He  is  currently  Lead  Math 
Modeler  for  the  F — 1 5  Simulated  Aircraft 
Maintenance  Trainers  (SAMTs).  He  holds  a 
Master's  Degree  from  the  United  States 
Naval  Postgraduate  School  in  Operations 
Research  and  was  a  former  Navy  Lieutenant 
Commander.  He  also  holds  a  Bachelor's 
Degree  from  Ohio  State  University  in 
mathematics.  He  was  formerly  a  consultant 
on  Space  Shuttle  at  Rockwell 
International.  Prior  to  this,  he  was  a 
Staff  Engineer  for  Bendix  Fielc 
Engineering  at  both  Goddard  Space  Flight 
Center  and  the  Jet  Propulsion  Laboratory; 
was  an  Operations  Research  Analyst  for 
NAVSEA;  and  was  a  member  of  a  ‘-mall  study 
team  working  directly  for  the 
Commander-in-Chief,  Atlantic  Fleet.  Mr. 
Stillings  has  published  in  the  fields  of 
Hydrofoil  Combat  Ships,  Nuclear  Submarines 
as  Task  Force  Escorts,  War  in  the 
Atlantic,  and  Random  Number  Generation  and 
Testing . 


Dr.  James  D.  Riley  is  a  Ser.  jcr 
Principal  Systems  Analyst  Engineer  with 
Honeywell  Training  and  Control  Systems 
Operations.  He  is  responsible  for  the 
engine  modeling  of  the  Simulated  Aircraft 
Maintenance  Trainers.  He  received  a 
Doctor's  Degree  in  mathematics  from  the 
University  of  Kansas.  He  was  formerly  a 
Senior  Staff  Scientist  at  McDonnell 
Douglas  Aeronautics.  Prior  to  this,  he 
taught  mathematics  at  several 
universities.  He  has  published  a  number 
of  papers  in  numerical  analysis. 


MSGT.  Mickie  T.  Wagner  is  F-100 
Engine  Training  Unit  Supervisor  and  Eng're 
Instructor  with  the  Field  Trair.irg 
Detachment  at  Hill  AFB.  He  has  been 
associated  with  the  F-100  engine  sirce 
1972  and  has  attended  both  the  McDonnell 
Douglas  and  the  Pratt  A  Whitney  Engjr.e 
Schools.  He  has  taught  the  F-100  Engjre 
since  1978. 


THE  FEASIBILITY  OF  EMPLOYING  AN  IN-COCKPIT 
DEVICE  TO  PROVIDE  MOTION  CUES  TO  THE  PILOT 
OF  A  FLIGHT  SIMULATOR 
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ABSTRACT 

A  study  was  undertaken  to  investigate  the  feasibility  of 
providing  motion  simulation  with  an  in-cockpit  device  rather 
than  an  external  motion  platform.  The  conventional  wisdom  has 
deemed  that  it  would  not  be  feasible  to  provide  the  necessary 
stimulation  of  the  vestibular  apparatus  because  of  insufficient 
excursion  inherent  in  an  in-cockpit  device.  This  paper  addres¬ 
ses  that  issue  in  light  of  recent  research  that  begins  to  clar¬ 
ify  this  interrelationship  between  the  visual  and  vestibular 
systems  in  the  perception  of  motion.  A  novel  approach  is  sug¬ 
gested  which  relies  heavily  orr  the  coordination  of  the  vi-aual 
and  vestibular  systems.  In  addition,  experimental  protocols 
are  suggested  by  which  the  approach  can  be  verified.  This 
study  was  originally  performed  for  helicopter  simulators  but 
the  technique  is  applicable  to  fighters  as  well  and  perhaps 
even  to  transport  aircraft. 


with  other  systems  and  training  value 
obtained. 

The  motion  platform  most  widely  used 
consists  of  a  hydraulically  powered  six- 
post  synergistic,  computer-driven  system 
designed  to  carry  and  provide  motion 
cues  to  a  cockpit  that  often  includes  a 
visual  system  with  electronics  and  a 
display  system  and  a  pilot.  Such  a  load 
requires  a  large,  powerful,  responsive, 
and  safe  system  capable  of  providing 
onset  and  sustained  acceleration  cues 
that  are  synchronized  and  compatible 
with  all  other  systems. 

The  initial  cost  of  o  typical  motion 
system  is  $400, 000.  If  a  ten  year  life 
span  at  16  hours  per  day  and  250  days 
per  year  and  10%  per  year  inflation  rate 
is  assumed  and  considering  costa  of 
spares,  training,  facilities,  main¬ 
tenance  and  the  like,  a  total  10  y^ar 
life  cycle  cost  of  $3.7  million  is  pre¬ 
dicted  using  a  lift  cycle  cost  model. 
This  leads  to  a  cost  of  $02. 50/hour  to 
operate  the  motion  system. 

The  safety  aspects  of  motion  systems 
must  be  carefully  considered  because 
they  use  high  pressure  hydraulics  and 
powerful  rams  to  create  the  motion  that 
drives  the  cockpit. 

Many  safety  features  must  be  in¬ 
corporated  to  preclude  damage  or  in¬ 
jury.  This  roejuires  hydraulic  and 
electronic  safety  systems,  and  more  and 
better  maintenance  facilities  which 
contribute  to  the  complexity  and  cost. 
Although  over  the  yoars_the  systems  have 
a  proven  safety  record"  and  are  docu- 


INTRODUCTION 

Since  motion  is  perceived  through 
stimulation  of  several  physiological 
receptor  systems  {visual,  vestibular, 
haptic  and  auditory),  many  designers  and 
users  of  flight  training  simulators  have 
felt  it  was  necessary  to  utilize  plat¬ 
form  motion  systems  to  provide  the  ves¬ 
tibular  ana  some  of  the  haptic  stimuli. 
This  has,  by  no  means,  teen  a  unanimous 
point  of  view.  In  fact,  the  subject  has 
been  controversial  in  the  flight  train¬ 
ing  community  for  quite  some  time. 

The  reasons  for  the  controversy  are 
manifold,  not  the  least  of  which  is  cost 
--  including  purchase,  facilities  and 
maintenance.  Other  factors  are  com¬ 
patibility  with  certain  visual  displays, 
false  cues  in  certain  maneuvers,  safety 
and  reliability. 

There  is  then  a  great  deal  of  moti¬ 
vation  to  eliminate  platform  motion  sys¬ 
tems.  It  is  our  opinion,  however,  that 
in  order  to  provide  the  appropriate  mo¬ 
tion  stimuli,  the  platform  motion  system 
must  be  replaced  with  an  adequate  surro¬ 
gate.  This  opinion  is  based  in  the  re¬ 
cent  research  which  indicates  some  ves¬ 
tibular  stimulation  is  necessary  for  the 
proper  percept  of  self  motion,  vll) 

PLATFORM  MOTION  SYSTEMS 

Motion  platforms  are  difficult  to 
justify  in  terms  of  the  more  obvious 
factors  such  as  initial  cost,  safety, 
facilities,  maintenance  and  life  cycle 
costs  but  also  in  terms  of  the  more 
subtle  factors  relating  to  integration 
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In  a  simulator ,  a  w i d c - a ng le  visual 
system  provides  stimulation  of  the  eyes* 
peripheral  sensors  which  results  in  a 
greater  visual  perception  of  motion. 
However,  the  perception  delay  is  still 
significant. 


With  the  vestibular  system,  tin* 
pilot  van  get  advanced  warning  of  a 
position  change  in  less  time  and  mag¬ 
nitude  of  position  change  than  required 
with  visual-only  cuing.  A  motion  plat¬ 
form,  however,  iias  limited  cuing  capa¬ 
bility  due  to  li:  ited  travel  but  as  long 
as  the  acceleration  cue  is  long  and 
large  enough,  and  the  cue  is  synchron¬ 
ized  with  the  visual,  a  faster  cue  is 
provided . 

This  is  not  very  easy  to  do  con¬ 
sidering  multiprocessor  configurations, 
throughput  lags,  iteration  rate  effects, 
hardware  response  times  and  all  the  fac¬ 
tors  contributing  to  mismatch  between 
the  visual  and  the  motion;  while  pro¬ 
viding  adequate  cuing  to  achieve  the 
desired  purpose. 

If  we  consider  the  task  of  syn¬ 
chronizing  the  platfoxm  and  the  visual 
systems,  there  are  many  considerations 
in  specifying  the  platform  performance. 
The  platform  must  provide  a  range  of 
acceleration  cues  of  acceptable  time 
duration  and  magnitude  with  subliminal 
acceleration  washout  within  the  platform 
excursion  limitations  while  reflecting 
visual  position  magnitude.  The  through¬ 
put  lags  and  computer,  hardware/software 
lags  must  be  within  an  acceptable  range 
of  the  visual  lags  so  as  not  to  be  un¬ 
acceptably  out  of  phase;  and  the  high 
response  high  pressure  hydraulic  hard¬ 
ware  should  not  respond  to  computer 
iteration  rate  stepping. 

The  existence  of  these  cue  synchron¬ 
ization  problems  is  known  to  be  a  vari¬ 
able  depending  on  magnitude  and  fre¬ 
quency  of  input  as  well  as  system  design 
and  has  been  quantitatively  documented 
in  a  few  cases .  (3) 


In  cases  oL  low- frequency  maneuver¬ 
ing  in  slow  aircraft  or  slow  maneuvering 
in  fast  aircraft,  the  delay  may  oo  ac¬ 
ceptable  . 

However,  previous  research  indicates 
that  high-frequency  disturbance-type 
tasks,  abrupt  failure  or  turbulence 
effects,  or  flight  dynamics  of  unstable 
aircraft  operating  in  specific  modes, 
such  as  helicopter  hover,  visual-only 
cuing  is  not  fast  enough  to  generate 
what  is  deemed  good  pilot  performance. 
Motion  improves  performance.  ^9) 

Abrupt  motions  which  generate  large 
acceleration  over  a  small  position 
change,  are  sensed  better  when 
acceleration  forces  are  provided  to 
stimulate  the  vestibular  sensors. 

The  vestibular  sensors  are  basically 
acceleration  sensors  and  (in  a  perfectly 
synchronized  simulator  or  the  real 
world)  respond  to  the  second  derivative 
of  position.  Therefore,  their  cuing 
information  is  significantly  faster  than 
the  visual  position  information. 


PSYCHOPHYSICAL  ASPECTS 

It  is  well  known  that  the  human  per¬ 
ceives  his  motion  through  space  by  the 
integration  of  information,  by  the 
brain,  from  the  transducers  of  several 
physiological  sensory  systems;  visual, 
vestibular,  haptic  and  aural.  The 
manner  in  which  the  apparatuses  of  these 
systems  operate  is  described  in  several 
sources.  11'  5,  7»  8,  14,  15)  jn 

addition  to  these  there  are  many  other 
sources  to  which  the  reader  may  refer  in 
order  to  obtain  complete  discussions  of 
the  sensory  systems.  This  information 
is  processed  yielding  a  usually  coherent 
perception  of  the  motion  environment. 

The  perception  is  not  always  coherent 
because  in  some  cases  information  is 
ambiguous  arid  in  others  specific  sensory 
information  is  absent. 

An  example  of  a  perceptual  ambiguity 
is  the  case  of  the  familiar  "railroad 
station  paradox."  In  this  instance,  a 
person  sitting  in  a  train  stopped  at  the 
station  perceives  relative  motion  with 
respect  to  an  adjacent  train  as  that 
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train  bog i ns  ' o  move.  His  initial 
reaction  in  that  the  train  in  which  he 
it  a  passenger  is  moving  in  the  opposite 
a  1  r  ec  t:  1  ori  of  the  train  that  actually  is 
moving.  Subsequently,  however,  he 
resolves  this  ambiguity  on  the  basis  of 
further  perceptual  information.  Hence, 
he  finally  concludes  that  it  is  the 
other  train  that  is  moving. 

This  visual  illusion  is  known  as 
vection.  Vection  is  the  perception  of 
self  motion  due  to  stimulation  of  the 
visual  system  particularly  in  the 
periphery. 

An  example  of  incoherent  perceptual 
information  arising  from  the  absence  of 
perceptual  information  or  incorrect  per¬ 
ceptual  information  would  be  an  airplane 
making  a  coordinated  turn  in  a  cloud 
Lank.  If  the  pilot  does  not  look  at  his 
instruments,  he  has  no  indication  that 
the  airplane  is  changing  heading.  This 
phenomenon  results  from  the  fact  that 
due  to  the  cloud  bank,  there  is  no 
visual  perception  of  heading  change  and 
since,  in  a  coordinated  turn,  the  forces 
are  biilaneed,  there  is  no  stimulation  of 
the  vestibular  or  haptic  apparatus.  In 
this  case,  there  is  no  knowledge  of 
heading  change  and  the  situation  may 
never  be  resolved  without  additional 
sensory  information  such  as  visual 
information  from  the  instruments. 

The  modern  flight  simulator  with  its 
wide-angle  visual  system,  moving  plat¬ 
form,  G-seat,  anti-G-suit  system  and 
vibration  systems  attempts  to  stimulate, 
to  some  extent,  all  of  the  above  men¬ 
tioned  sensory  systems.  In  general, 
motion  base  systems  employ  an  onset  cue 
with  a  washout  to  provide  an  initial 
stimulus  to  the  vestibular  system  and 
haptic  system.  Subsequently,  the  visual 
system,  relying  upon  the  vection  phe¬ 
nomenon,  sustains  the  motion  sensation. 
G-suits  a:.d  C-seats,  both  stimulating 
the  haptic  system,  are  employed  to 
augment  the  vection  phenomenon  for 
sustained  motion  and  to  provide  some 
hiyh-G  cuing.  Vibration  systems  are 
used  to  provide  the  high  frequency  (up 
to  40  Hz,)  oscillatory  motion  cues. 

The  major,  as  yet  unanswered,  ques¬ 
tion  in  flight  simulation  today  is:  "Is 
platform  motion  necessary  in  the  pres¬ 
ence  of  wide  field  of  view  visual  sys¬ 
tems  V"  One  point,  of  view  states  that 
since  self  motion  can  be  induced  visu¬ 
ally.  there  is  no  need  for  platform 
motion  systems.  Another  point  of  view 
is  founded  in  the  research  of  Young  and 
others,  which  basically  states  that 
the  onset  of  vocticn  is  hastened  by 
the  presence  of  vestibular  stimula¬ 
tion.  ^11'  13}  Brandt  and  others 
state  that  v.ithout  vest  i  ou.lai  stimu¬ 
lation  the  onset  of  voetj.ori  can  be  of 
the  order  of  2  to  3  2  seconds. 

However,  with  the  addition  of  vestibular 
stimulation  this  onset  delay  was  im¬ 


perceptible.  (2D  These  resoHrcliere 
have  also  found  that  the  direction  of 
the  stimulus  is  more  important  than  the 
magnitude.  Implicit  in  this,  of  course, 
is  the  fact  that  the  stimulus  must  be  in 
excess  of  the  perceptual  threshold. 

Young  offers,  "Merely  a  slight  platform 
displacement  or  seat  motion  in  the  ap¬ 
propriate  direction  may  be  sufficient 
to  bring  forth  well  developed  vec¬ 
tion."  ^  1 2 ) 

It  is  safe  to  say,  then,  that  there 
is  sufficient  evidence  that  vection 
takes  too  long  to  develop  in  the  absence 
of  vestibular  stimulation  to  be  effec¬ 
tive  in  the  maneuvering  environment. 
Secondly,  vection  is  quickened  by  ves¬ 
tibular  stimulation.  Thirdly,  small 
amplitude  seat  motion  in  the  correct 
direction  will  quicken  the  onset  of 
vection.  The  foregoing  is  encouraging, 
in  that  it  appears  to  offer  a  suitable 
basis  for  justification  of  a  motion  seat 
in  lieu  of  a  motion  platform. 

CUING  APPROACHES 

There  are  throe  possible  cuing 
approaches  which  could  be  investigated 
here.  The  first  is  to  employ  an  in¬ 
cockpit  devicf-  in  uii  onset  cuing  mode  as 
is  commonly  done  with  platform  motion 
systems.  In  this  mode,  the  seat  would 
be  used  to  stimulate  both  vestibular  and 
haptic  systems.  Onset  and  washout  drive 
logic  would  be  employed  in  this  case.  A 
second  approach  is  to  employ  the  seat  to 
stimulate  solely  the  haptic  system  in  a 
sustained  cuing  modality.  The  third 
approach  is  to  use  the  seat  to  provide  a 
very  sliort  duration,  pulse-type  cue  in 
the  appropriate  direction. 

Onset  Cuing  Approach 

In  order  to  provide  onset  cuing  with 
washout,  a  substantial  amount  of  seat 
pan  displacement  is  required.  For  ex¬ 
ample,  to  provide  a  vertical  onset  cue 
for  0.2  seconds  with  a  maximum  ac¬ 
celeration  of  1 . 2g  anti  employing  a 
linear  washout  which  is  below  the 
indifference  threshold  given  by  Hoffman 
and  Reidel  as  O.lg,  the  excursion 
required  would  be  30.9  inches. 

This  is  obviously  too  great  for  an 
in-cockpit  device.  It  should  be  noted 
that  shortening  the  duration  of  the  cue 
wi'll  serve  to  reduce  3Ubstant ia lly  the 
total  displacement  required.  For  ex¬ 
ample,  if  the  cue  duration  is  reduced  to 
0.1  seconds  tor  the  onset,  the  system 
excursion  requirements  are  reduced  by  a 
factor  of  four. 

Therefore,  reducing  the  onset  cue 
duration  to  0.1  would  enable  the  pre¬ 
sentation  of  a  O.Jg  cue  and  associated 
acceleration  washout  in  0.63  inches  of 
platform  excursion.  This  would  br 
feasible  within  the  context  of  most 
existing  G-ncats  and  also  bo  acceptable 
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within  the  constraints  imposed  by  the 
cockpit  geometry.  However,  the  cuing 
dynamic  range  would  be  small,  i.e.,  from 
O.lg  to  O.dg.  An  onset  cue  of  0.1 
seconds  would  be  near  the  upper  end  of 
the  time  constant  range  of  the  otoliths, 
which  is  loss  tiian  0.1  seconds. 

Examining  the  rotut lone  1  n\''s  in  the 
same  manner,  and  considering  a  rota¬ 
tional  velocity  cue  required  c-f  20°/ 
sec.,  it  was  found  that  tnis  cue  would 
require  over  seat  pan  rotation  an.: 

the  resulting  washout  velocity  of 
10°/ sec.  is  amove  the  indifference 
threshold  of  2°. 'sec.  These  values  are 
based  on  a  washout  acceleration  of 
5°/sec.^  and  a  cue  duration  of  0.2 
seconds.  The  40°/sect  pan  rotation 
results  in  5. 2d  inches  of  vertical 
displacement,  assuming  a  12  inch 
separation  of  seat  actuators.  As  was 
the  case  for  the  vertical  excursion 
case,  the  cue  duration  could  be  reduced 
and  this  would  yield  lower  performance 
requirements.  However,  considering  tine 
constants  of  1  to  10  seconds  for  the 
semi-circular  canals,  then  a  cue 
duration  of  0.3  sec.  would  oe  small  as 
compared  to  the  time  constant.  (!->' 

It  should  be  noted  that  linear  cuing 
only  was  considered  above  and  if  non¬ 
linear  drive  models  were  developed,  the 
situation  could  imprc  e.  However,  it  is 
unlikely  it  would  improve  to  the  ext**  *■ 
required  for  credible  simulation. 

Haptic  System  Stimulation  Approach 

The  question  hers  is  whether  using  a 
device  like  a  G-seat  to  stimulate  the 
pressure  and  muscle  receptors  of  the 
haptic  system  wou!.d  be  a  viable  approach 
in  the  context  of  this  investigation. 

The  traditional  application  of  G-seats 
in  flight  simulators  has  been  to  employ 
them  together  with  platform  motion  sys¬ 
tems  to  provide  the  sustained  accel¬ 
eration  cues  while  the  platform  provides 
the  vestibular  stimuli  via  onset  cuing. 
The  sustained  cues  have  been  provided  by 
altering  the  pressure  distribution, 
across  the  back  and  buttocks,  due  to 
pilot  weight  and  aircraft  acceleration 
vector.  The  general  contribution  of 
haptic  system  stimulation  is  quite  well 
recognized  at  this  point.  However,  what 
is  not  known  is  the  relative  importance 
of  the  two  stimuli  n  the  perception  of 
motion.  Further,  no  research  has  been 
performed  to  differentiate  the  relative 
importance  of  the  two  stimuli  on  the 
development  of  vection.  However,  it  is 
known  that  some  elements  of  the  haptic 
system  have  small  displacement  thres¬ 
holds  and  relatively  low  time  constants 
(Table  1).  This  leads  one  to  believe 
they  might  be  stimulated  by  a  small 
excursion  system. 


TABLE  (1) 


Sensor 

Dl  Sp  lace  merit 
Tnresnol Is* 

T  i  me 

Const  -int 

I'joinun  Corpuscles 

McfKi-j’s  Discs 

Kutfini  Emi  Jr  joins 

'1  jsc  1 1*  S^ini'es 

10  microns 

1-5  raai 

1 - 1 0  ms 

1  i  s 

1,  5,  i,  20  s 

80  m.= 

♦These  data  from  Borah,  et  al. 


These  data  offer  ^me  reason  for 
optimism  in  that  tlie  thresholds  are  very 
low  for  the  cutaneous  and  subcutaneous 
sensors,  hence  pressure  stimuli,  from  a 
variable  contour  seat  pan,  woujd  aid  in 
the  perception  of  motion  via  direct  per¬ 
ception  and  perliaps  through  the  quicken¬ 
ing  of  vection. 

A  G-seat  type  device  altering  the 
pressure  distribution  of  the  back  and 
buttocks  could  be  used  in  conjunction 
with  some  small  onset  device  as  dis¬ 
cussed  in  the  previous  section.  How¬ 
ever,  this  adds  complication  to  the 
system. 


As  stated,  one  related  area  which 
has  not  been  pursued  is  the  effects  cf 
haptic  stimulation,  in  the  absence  of 
vestibular  system  stimuli,  on  the  onset 
of  motion  perception  induced  by  vec¬ 
tion.  However,  Young  states  that  tac¬ 
tile  cues  respond  most  rapidly  to 
changes  in  pressure  and  signal  any  rapid 
changes  in  acceleration  because  of  the 
consequent  change  in  support  force  "They 
are  the  ideal  first  simulator  cue  for 
rapid  onset."  (12)  This  is  borne  out 
by  the  data  of  Table  (1). 

Pulse  Cuing  Approaclt 

The  Pulse  Cuir.g  approach  has  its 
justification  in  the  research  of  several 
investigators  who  have  indicated  that 
the  vection  phenomenon  is  hastened  with 
vestibular  stimulation,  (2»  13) 

and  also  that  the  vestibular  stimulation 
r.eed  only  be  a  short  pulse  in  the 
appropriate  direction.  (I2) 

While  this  approach  might  involve  a 
multi-degree-oc-f reedom  seat  platform  in 
the  cockpit,  it  would  have  very  small 
excursions.  How  many  degrees  of  freedom 
are  required  must  be  determined  by  addi¬ 
tional  research.  At  least  four  degrees 
of  freedom  are  required,  the  three  ro¬ 
tational  and  vertical.  It  is  probable 
that  linear  vection  can  be  quickened 
with  rotational  vestibular  stimuli,  but 
this  must  be  verified.  Another  ad¬ 
vantage  of  this  approach  is  that  the 
software  drive  algorithm  would  be  very 
simple.  Also,  since  the  excursions 
would  be  small,  eyepoint  motion  would  be 
small  and  hence  dynamic  mapping  of  the 
eyepoint  would  not  be  necessary.  Be¬ 
cause  of  the  small  excursion  require¬ 
ments  a  pneumatic  system  may  satisfy  the 
response  requirements. 
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The  vertical  decree  of  freedom 
should  also  be  capable  of  providing 
vibration  cues  in  the  range  of  3  to  40 
Hz. 

SUGGESTED  RESEARCH 

In  order  to  evaluate  the  validity  of 
the  candidate  approaches  further  re¬ 
search  is  needed.  The  research  falls 
into  two  categories,  that  which  will 
provide  more  psychophysical  data  and 
that  which  will  verify  simulation 
concepts.  Some  examples  of  the  first 
category  are:  Will  haptic  system 
stimulation  alone  hasten  the  onset  of 
vection?  Will  rotational  vestibular 
stimuli  quicken  linear  vection?  What 
are  the  required  pulse  characteristics, 

i.e.,  pulsewidth  and  range  of  magnitudes? 

A  prob’em  with  much  of  this  research 
is  that  of  finding  an  objective  measure 
of  the  results.  In  this  case,  using 
visual  nystagmus  should  provide  the 
desired  results.  Other  techniques  are 
also  available.  One  technique  for 
obtaining  quantitative  objective  measure 
of  psychophysical  phenomena  is  the 
method  of  evoked  response.  Much  has 
been  written  in  the  literature  about 
visual  evoked  potential  (VEP)  but  to  our 
knowledge,  this  technique  lias  not  been 
applied  to  determining  the  onset  of  vec¬ 
tion,  or  for  that  matter  other  simula¬ 
tion  research. 

The  second  category  of  research  is 
more  simulation  specific.  Some  examples 
of  the  kinds  of  research  in  this  cate¬ 
gory  are:  employ  a  one-dimensional 
tracking  task  both  with  and  without  the 
pulse  cue  to  hasten  tn  onset  of  vec¬ 
tion,  which  would  be  stimulated  by  an 
appropriate  display.  The  subjects' 
performance  in  both  cases  would  be 
measured  and  compared.  The  experiment 
should  also  be  performed  using  an 
unconstrained  single-deyree-of- freedom 
rotational  motion  system  in  place  of  the 
pulse  cue.  The  subject's  performance  on 
this  task  would  then  be  compared  with 
the  other  two  conditions.  Other  trans¬ 
fer  of  training  experiments  may  be  per¬ 
formed  as  well. 

Needless  to  say,  subjective  evalua¬ 
tions  with  experienced  pilots  must  be 
performed  as  well,  in  order  to  ensure 
that  the  technique  will  gain  acceptance 
in  the  user  community.  Also,  experi¬ 
mentation  with  various  drive  algorithms 
is  necessary.  In  the  case  of  the  pulse 
cuing  approach,  the  only  aspects  of  the 
drive  algorithm  to  be  considered  are  the 
magnitude,  duration  and  direction  of  the 
pulse . 

CONCLUSION 

Of  the  three  approaches  suggested  it 
appears  that  the  option  with  the  most 
promise  is  the  pulse  cuing  approach. 


The  onset  cuing  approach  has  very  little 
expectation  of  success  because  of  the 
limited  stroke  available.  There  is  some 
question  as  to  whether,  even  in  large 
stroke  motion  platforms,  an  optimum 
drive  algorithm  has  been  developed. 
Hence,  the  degree  of  acceptability  is 
very  low  for  these  systems.  It  is, 
therefore,  unreasonable  to  expect  that 
an  in-cockpit  device  with  much  less 
stroke  would  ever  be  acceptable. 

The  second  approach  employing  haptic 
system  stimulation  does  have  some  prom¬ 
ise  but  requires  a  substantial  amount  of 
research  for  this  application.  In  addi¬ 
tion,  the  effort  to  develop  suitable 
dri^e  algorithms  would  be  extensive. 

The  pulse  cuing  approach  is  attrac¬ 
tive  in  its  simplicity  as  well  as  the 
reasonably  high  expectation  of  success 
it  engenders. 
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PROTOTYPE  SPECIFICATIONS  TO  SUPPORT  HIGH 
RESOLUTION-SENSOR  SIMULATORS 


Ronald  J.  Pierce 

Defense  Mapping  Agency  Aerospace  Center 
St.  Louis  AFS ,  MO  63118 


ABSTRACT 

^he  Defense  Mapping  Agency  is  developing  a  follow  on  Prototype  Product  Specification 
for  Digital  Data  to  support  High  Resolution  data  requirements  in  the  DoD ,  The  computer 
hardware  technology  is  growing  faster  than  the  software  technology  to  enable  simulators 
to  portray  realism.  Both  these  technologies  and  the  DoD  requirements  are  growing  much 
faster  than  DMA's  ability  to  produce  the  digital  products  required  for  the  multiple 
applications.  The  Prototype  specification  is  an  effort  to  narrow  the  gap  of  these 
technologies . 


“'This  paper  will  discuss  the  development  of  the  High  Resolution  data  Specifications; 
define  and  compare  parameters  of  the  current  and  Prototype  data;  describe  the 
geographical  area  of  coverage;  and,  discu  s  urei  evaluation  and  validation  of  the  data 
content. 

\  Introduction 


The  Defense  Mapping  Agency  has  been 
producing  digital  data  since,  the  early 
1960's.  This  data  was  generally  used  to 
produce  relief  Maps.  It  wasn't  until  1972 
that  DMA  was  directed  to  produce  digital 
data  to  support  an  R&D  effort  to  provide 
radar  displays  in  simulators  for  aircrew 
training.  A  production  specification  for 
digital  data  for  Radar  simulation 
resulted  in  this  R&D  effort.  This 
specif ication  was  published  in  1974  and 
is  called  Digital  La^dmass  system  (DLMS) . 
The  DLMS  product  is  composed  of  a  Matrix 
of  terrain  elevations  called  Digital 
Terrain  Evaluation  Data  (DTED )  and 
Cultural  data  called  Digital  Feature 
Analysis  Data  (DFAD) .  DFAD  predicts  the 
radar  reflectivity  of  the  earth's 
surface.  The  DLMS  program  supoorts 
Weapons  Systems  Trainers  (WST)  and 
operational  uses  sucl  as  cruise  missile 
planning,  Firefinder,  aircraft  mission 
planning,  etc.  The  DLMS  Program  requires 
millions  of  square  nautical  miles  of  DFAD 
and  DTED. 


changes  in  these  revisions  since  1977 
are: 

a.  Standardization  of  feature 
descriptors  -e.g.,  Surface  Material  Code 
(SMC)  3  Farm  structures  portrayed  as 
point  features. 

b.  Increase  feature  identification 
codes  -e.g.,  from  57  to  over  260. 

c.  Decrease  minimum  size  for  strong 
reflectors  -e.g.,  Bridge. 

d.  Increase  minimum  size  for  poor 
reflectors  -e.g.,  Desert  areas. 

e.  Expansion  of  unique  features  - 
e.g..  Railroad  Gantr ies/Py Ions . 

f.  Standardization  of  percent  of 
roof  cover  descriptor  -  e.g.f  0%,  10%  and 
30%. 

g.  Portrayal  of  permanent  snow  and 

ice. 

Another  change,  2nd  Edition  DLMS,  is 
currently  in  coordination  which  will 
include  the  addition  of  LOC ' s  (lines  of 
communication) . 


In  1975  DMA  was  given  the  responsibility 
to  produce  all  the  Mapping,  Charting  and 
Geodesy  (MC&G)  digital  data  required  by 
the  DoD.  With  this  responsibility  it  was 
necessary  to  insure  that  all  users 
requirements  were  reflected  in  the 
specification.  As  experience  and  user 
feedback  were  received  using  the  initial 
DLMS  specifications  DMA  refined  the 
specification  to  better  support  sensor 
simulation.  The  first  revision  of  the 
specification  was  in  July  1977,  with 
three  smaller  revisions,  changes  1,  2  and 
3,  in  1980-81,  resulting  in  the  current 
product  specification.  Some  of  the  major 


PHOTOTYPE  PRODUCT  SPECIFICATION  FOR 
HIGH  RESOLUTION  SENSOR  SIMULATION 

In  1978  DMA  received  an  Air  Force  request 
to  supply  digital  data  tc  support  an  out 
of  the  cockpit  visual  simulator.  Until 
this  request  was  received,  DMA  had  only 
been  supporting  radar  simulation 
requirements  and  the  specification  were 
designed  for  that  purpose  only.  With  the 
advent  of  visual  requirements,  it  became 
obvious  to  DMA  that  a  major  revision  to 
the  digital  data  specification  was 
required . 
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Investigations  were  begun  to  determine 
the  data  elements  required  for  visual 
simulation.  Discussions  with  various  DoD 
users  and  contractors  revealed  that  no 
one  had  the  same  requirement  for  MC&G 
support.  Recognizing  the  importance  of 
the  next  generation  cf  specifications, 
DMA  hosted  a  meeting  at  St.  Louis  in  Sept 
1978  for  interested  DoD  users  and 
contractors  to  take  part  in  preparing  a 
specification  for  visual  simulation. 
Personnel  in  attendance  represented  Hq 
Strategic  Air  Command  (SAC) ,  Hq  Military 
Airlift  Command  (MAC) ,  Hq  Tactical  Air 
Command  (TAC) ,  Pacific  Air  Forces 
(PACAF) ,  Naval  Training  and  Equipment 
Center  (NTEC ) ,  Aeronautical  Systems 
Division  (ASD) ,  Engineering  Topographic 
Laboratory  (ETL) ,  Rome  Air  Development 
Center  (RADC)  and  both  production  centers 
of  DMA,  Hydrographic/Topographic  Center 
(DMAHTC)  and  Aerospace  Center  (DMAAC) . 
The  first  meeting  resulted  in  a  decision 
to  develop  a  Prototype  Specification  for 
data  which  was  designed  to  support  high 
resolution  sensor  simulation  including 
Visual,  Synthetic  Aperature  Radar  (SAR) , 
Low  Light  Level  Television  (LLLTV)  and 
Infrared  (IR) .  The  prototype  data  will  be 
used  to  evaluate  how  well  high  resolution 
sensor  simulation  can  be  supported  by  aa 
enhanced  DLMS . 

This  paper  covers  only  the  planimetric 
(cultural  and  landscape)  features.  DMA 
believes  that  Standard  Level  I  Terrain 
will  be  satisfactory  for  high  resolution 
sensor  simulation. 

DRAFT  SPECIFICATION 

The  next  generation  specification  is  one 
that  must  satisfy  requirements  for  all 
services,  therefore,  DMA  developed  a 
policy  on  Future  Data  Bases  to  include  the 
following  major  points: 

1.  Common  data  elements  will  be 
established  for  multiple  applications. 

2.  The  data  will  be  stored  in  such  a 
way  as  to  permit  multiple  product 
generation . 

3.  The  format  will  be  designed  to 
support  evolving  requirements. 

4.  In  order  to  save  critical, 
manpower,  DMA  will  produce  to  the  most 
stringent  requirements  where  multiple 
products  are  scheduled. 

Utilizing  this  guideline  and  meeting  with 
the  various  users  and  their  support 
organizations,  a  prototype  product 
specif icat.ior.  was  prepared  and  published 
in  December  1979.  Five  areas  in  the 
continental  J.S.  were  selected  to  be 
compiled  to  this  specification  and 
production  was  initiated. 


SPECIFICATION  CHARACTERISTICS 

The  Prototype  Specification  consists  of 
geographically  defined  areas 

(manuscripts)  which  contain  two  separate 
levels  of  information.  This  information 
describes  the  geographic  location  and  the 
associated  descriptive  information 
defining  natural  and  man-made  features  on 
the  surface  of  the  earth  (similar  to 
current  DLMS)  .  Level  V  (for  visual) 
which  is  similar  to  Level  I  DLMS,  is 
comprised  of  relatively  large 

geographically  defined  areas  containing  a 
description  and  porti.ayal  of  natural  and 
man-made  features  presented  in  a 
standardized  digital  format.  Level  V 
data  is  designed  to  cover  large  expanses 
of  the  earth's  surface;  therefore,  it  is 
designed  to  contain  a  generalized 
portrayal  of  features.  Level  V  is  meant 
to  support  the  generation  of  high  detail 
by  using  Computer  Image  Generation  (CIG) 
and  Synthetic  Break-Up  (SBU)  techniques. 
CIG  and  SBU  are  processes  by  which 
descriptions  of  large  homogeneous  areal 
features  (regions)  are  used  with  computer 
software  to  break  features  into  component 
parts.  It  is  expected  that  these 
techniques  will  be  employed  by  the  user  as 
required. 

MAJOR  DIFFERENCES  BETWEEN  LEVEL  V 
AND  LEVEL  I 

1.  Level  V  includes  more  features. 

Level  V  specification  includes 

all  Line£  of  Communication  (LOC)  such  as 
roads,  railroads/power lines,  etc.,  and 
streams,  lakes  and  ponds.  Level  I  was 
developed  for  radar  simulation  and 
generally  contains  only  radar  significant 
features. 

2.  Level  V  utilizes  microdescriptors 

Microdescriptors  (feature 

attribute  attachment ) are  multipurpose 
descriptors  which  describe  additional 
(visual)  characteristics  of  a  feature. 
By  using  this  information  as  the  basis  for 
statistically  based  feature  generation,  a 
more  realistic  breakup  can  be  performed 
than  by  purely  random  means.  Four 
microdescriptors  were  developed  for  Level 
V. 

3.  Level  V  has  unique  Surface 
Material  Code  for  asphalt  (SMC  14) . 

Asphalt  was  included  in  SMC  9  in 
the  DLMS  specification. 

4.  Level  V  portrays  isolated 
structures  (SMC  4)  ncluding  composition 
structures  that  are  not  radar  significant 
but  ere  visually  significant. 

5.  Feature  Area  Code  (FAC)  #1,  of 
Level  V  is  the  most  predominant 
background  feature  in  the  given  area.  In 
the  DLMS  specification,  feature  #1  is 
always  normal  soil. 
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feature 


6.  No  feature  separation  criteria  is 
included  in  Level  V.  DLMS  has  an  areal 
feature  separation  of  500  feet.  Level  V, 
without  this  separation,  can  be  utilized 
to  generate  more  realistic  'scenes  by 
showing  continuous  streams/features 
regardless  of  width. 

7.  Level  V  portrays  wood 
obstructions  50-150'  high.  DLMS  portrays 
only  those  obstructions  over  150'. 

TERRAIN  ANALYSIS  DATA 

In  the  fall  of  1381,  DMA  was  asked  to 
expand  the  Level  V  format  to  accept  the 
data  elements  of  terrain  analysis  using 
DMAHTC's  Draft  Production  Specifications 
for  the  Tactical  Terrain  Data  Base 
(1:50,000)  dated  September  1981.  DMAAC 
created  three  additional  microdescriptors 
and  coordinated  with  HTC  and  Army 
personnel  on  common  data  elements.  An 
area  in  Ft.  Lewis,  Washington  was 
selected  as  a  test.  In  order  to  compare 
the  density  of  data  and  evaluate  content. 
Level  V  and  Level  V  enhanced  with  terrain 
analysis,  were  both  compile  1  over 
Ft.  Lewis. 

Figure  1  is  a  portion  of  the  manuscript  of 
the  Ft.  Lewis  area  compiled  to  the 
current  1977  DLMS  specifications.  Figure 
2  is  the  same  area  compiled  to  the  Level 

V  specifications  and  Figure  3  is  the 
Terrain  Analysis  Specifications.  The 
correlation  of  the  data  from  these  three 
specifications  is  extremely  close.  Level 

V  is  compatible  with  Level  1  but  defines 
the  features  in  more  detail.  The  terrain 
analysis  manuscript  is  compatible  with 
the  Level  V,  and  also  portrays  much  more 
detail.  Figure  4  shows  the  Level  V  and 
terrain  analysis  data  merged  and  plotted 
into  a  manuscript  for  both  Air  Force  and 
Army  programs.  This  is  the  new  Prototype 
High  Resolution  Specification  capable  of 
multiuse.  It  has  common  data  elements  and 
can  be  expanded  to  accept  new 
r equ i rements . 

DATA  DENSITY 

The  density  of  the  data,  as  well  as  the 
resources  required  to  produce  Level  V  or 
enhanced  Level  V,  can  increase  3-5  times 
over  the  current  Level  1.  In  order  to 
handle  this  magnitude  of  data,  and  have  a 
specification  that  is  as  flexible  to 
support  multiple  products,  three 
additional  microdescriptors  are  required. 
These  microdescriptors  plus  the  four 
developed  for  Level  V  can  be  used  when 
necessary  to  further  define  a  feature  to 
satisfy  a  specific  product  requirement. 
The  available  microdescriptors  are  as 
follows : 


1.  Vertically  composite 

(e.g.,  tower  on  a  building). 

2.  Homogeneous  area  descriptor 
(e.g.,  residential  area). 

3.  A  pattern  definition  (e.g., 

street  or  field  pattern) . 

4.  Combination  of  2  and  3. 

5.  Vegetation 

6.  Transportation 

7.  Surface  Drainage 

Figure  5  will  be  used  as  an  example  of 
data  stored  on  a  microdescriptor.  For 
purposes  of  this  example,  feature  number 
97  will  bt  utilized  (Table  1). 

The  Feature  Analysis  Data  Table  contains 
all  the  data  recorded  by  the  analysts  for 
this  feature.  The  top  line  is  the  primary 
descriptive  information  which  is 
determined  for  all  features  (Table  2)  and 
the  succeeding  lines  will  be 
microdescriptor  information,  if  required, 
feature  number  97  is  a  forest  area  and 
requires  the  microdescriptor  number  5  to 
satisfy  the  user  requirement  in  this 
geographic  area.  (Table  3) 

PROTOTYPE  TEST  AREA 

The  five  geographic  areas  selected  by  the 
users  for  testing  the  Level  V 
specification  are  as  follows: 

Area  1  -  A  rectangle  around 
Norfolk,  VA  and  NAS  Oceana  containing 
approximately  450  SNM. 

Area  2  -  A  15  mile  radius  circle 
centered  on  the  main  runway  at  Barksdale 

AFB ,  LA  (700  SNM). 

Area  3  -  An  area  around  Little 
Rock  AFB,  AR  containing  1850  SNM. 

Area  4  -  100  SNM  over  New  York 
City  covering  Manhattan,  Island. 

Area  5  -  Fallon,  Nevada.  An  area  of 
approximately  1400  SNM. 

These  five  areas  will  provide  the  user  all 
types  of  culture  and  landscape 
information  from  large  cities  to  desert 
areas  and  allow  them  to  evaluate  the 
adequacy  of  the  specification. 

An  additional  area,  No.  6,  covering  Ft. 
Lewis,  Washington,  has  been  produced 
containing  both  Level  V  and  terrain 
analysis  data.  This  area  contains  250 
SNM. 

Other  areas  3re  being  compiled  to  the 
terrain  analysis  enhanced  Level  V 
specification  and  should  be  completed  by 
the  end  of  1982. 
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TABLE  1 


FEATURE  ANALYSIS  DATA  TABLE 
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CODED  DESCRIPTIVE  DATA  FOR  FT.  LEWIS,  WASHINGTON 


TABLE  2 

FEATURE  ANALYSIS 
DATA  TABLE  (FADTP) 
PRIMARY  DESCRIPTOR 

*  FAC  NUMBER- 97 

'  FEATURE  TYPE- 2 

*  FEATURE  IDENTIFICATION  CODE- 9531 

*  SURFACE  MATERIAL  CATEGORY- 12 

*  PREDOMINANT  HEIGHT- 18 

*  NUMBER  OF  MICRODESCRIPTORS- 1 


TABLE  3 

VEGETATION 
MICRODESCRIPTOR  #5 

FAC  NUMBER- 97 
MICRO  TYPE- 5 

CANOPY  (SUMMER)  CLOSURE- 3 
STEM  DIAMETER- 9 
STEM  SPACING- 11 
VEGETATION  ROUGHNESS  FACTOR- 4 
UNDERGROWTH- 2 
TREE  CROWN  DIA-0 
HEIGHT  OF  LOWEST  BRANCH- 0 


SPECIFIC  TYPE- 3 
QUALIFIER-1 
STATE  OF  GROUND- 1 
DEPTH  OF  SMC  ROUGHNESS- 2 
(MED  &  HEAVY  TANKS) -8 
ROUGHNESS  (LARGE  WHEEL  VECH) -0 
ROUGHNESS  (WHEEL  VECH)-0 
ROUGHNESS  (FOOT  TROOPS)-0 
CLOSURE  RATED  CONE  INDEX- 0 
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Figure  5  -  Ft.  Lewis,  Washington  Manuscript  'Areal  Features) 


SUMMARY 

Six  domestic  areas  have  been  compiled 
using  the  new  specification.  These  areas 
total  approximately  5000  square  nautical 
miles.  The  areas  will  be  made  available 
to  the  users  for  test  and  evaluation  of 
the  data  content.  DMA  will  be  requesting 
comments  on  the  specification  as  to  the 
ability  to  satisfy  current  and  future 
requirements.  ASD  is  working  with 

interested  simulator  contractors  and  is 
soliciting  their  comments.  The  Federal 
Republic  of  Germany  and  the  United 
Kingdom  have  also  indicated  interest  in 
the  new  specification.  A  copy  of  the 
specification  and  a  magnetic  tape  of  the 
Ft.  Lewis  area  is  being  provided  these 
NATO  countries  for  their  evaluation  and 
comments . 

An  evaluation  plan  is  presently  being 
developed  by  a  committee  headed  by  HQ  DMA 
with  members  from  each  service,  DMAHTC 
and  DMAAC .  The  purpose  of  this  plan  is  to 
establish  milestones  to  insure  that  all 
comments  are  considered  in  the  revision 
of  the  final  specification.  The  goal  is 
to  have  a  specification  that  will  support 
an  all  purpose  multiuse  digital  data  base 
in  1985. 
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ABSTRACT 

\ 

V 

*  Training  simulators  for  modern  analysis  sonar  and  sonobuoy  processing  equipments 
demand  high  accuracy  target  generators  for  maximum  training  value.  The  essential  requirement 
is  for  high  frequency  stability  and  resolution  and  a  wide  range  of  frequencies,  whether 
these  be  representing  discrete  signals  or  forming  modulating  envelopes.  This  requirement  is 
due  to  the  high  resolution  of  such  nars  where  any  instability  or,  steps  in  the  generated 
signals  show  on  the  analysed  signal  a  a.;  can  confuse  trainees.  A  ^Passive  Acoustic  Target41-' 


module  has  been  developed  which  satisfies 

features  and  the  method  of  realisation  of 

INTRODUCTION 

Requirement 

The  equipment  described  in  this  pape*"  was 
designed  and  produced  to  meet  a  training  require¬ 
ment  identified  by  various  users.  The  requirement 
was  to  produce  sonar  simulators  for: 

1.  Advanced  Operator  Training  in  the  use 
of  sophisticated  analysis  sonars  and  sonobuoy 
processing  systems. 

2.  Combined  Tactical  Training  at  an 
advanced  level. 

The  level  of  instruction  which  was  to  be 
given  was  identified  by  the  potential  instructors 
as  requiring  the  use  of  operational  equipment 
which  would  respond  to  the  operators  controls 
with  a  high  degree  of  realism.  The  use  of  fac¬ 
simile  equipment  was  considered  not  to  be  accep¬ 
table  for  the  high  level  of  training  to  be  done. 

It  is  therefore  clear  from  the  above 
constraints  that  it  was  necessary  to  produce  a 
sysf.em  which  would  stimulate  sufficient  of  the 
operational  equipment  to  satisfy  the  requirement. 
It  will  be  appreciated  that  in  producing  such  a 
stimulus  to  operational  equipment,  which  is  going 
to  produce  long  term  hard  copy  of  the  analysed 
signals,  both  long  term  and  short  term  insta¬ 
bility  need  to  be  abrent  in  these  analysed 
signals  except  for  ones  wnich  form  a  part  of  the 
target  vessel  signature,  and  the  resolution 
of  the  generator  must  be  high  so  that  st«  ps  in 
the  analysed  waveform  cannot  be  seen. 

Stimulation  Signals 

The  basic  method  of  producing  a  stimulation 
signal  for  a  sonar  system  is  well  known  and  basic¬ 
ally  consists  of: 

1.  Generation  of  Target  Produced  noise. 

2.  Generation  of  background  noise  to 
represent  the  environment  in  which  the  target 
exists. 

1.  The  combination  of  these  two  component j 
at  the  correct  level  ar  the  transducer  of  the 
simulated  sonar  taking  into  account  source 
strengths  and  propagation  losses. 


the.  above  requirement.  The  paper  describes  the 
modules  to  satisfy  the  requirement..* 

-"I 

4.  Processing  the  signals' to  represent 
those  parts  of  the  sonar  system  which  are  not 
used  in  the  simulator. 

5.  Injection  of  these  combined  and  pro¬ 
cessed  signals  into  the  operational  equipment  at 
the  chosen  point. 

The  only  item  to  be  considered  in  this 
paper  is  the  generation  of  the  target  produced 
noise . 

PASSIVE  ACOUSTIC  TARGET  (PAT)  MODULE 
PAT  Components 

A  block  diagram  of  the  PAT  module  is  s^own 
in  Fig.l.  The  components  of  the  PAT  are: 

1.  An  Intelligent  Interface  Module  (IIM) 
consisting  of  a  micro, rocessor  with  associated 
PROM  and  RAM  to  which  data  from  the  main  sim¬ 
ulation  computer  is  sent  defining  the  targets  to 
be  generated.  This  data  is  processed  to  generate 
the  control  signals  and  data  for  the  remainder  of 
the  nodule. 

2.  An  Inverse  Fourier  Transform  Module 
(IFTM)  which  generates  accurately  the  required 
waveforms.  This  is  described  in  detail  later. 

3.  Analogue  processing  circuitry  utilising 
the  accurately  generated  waveforms  to  produce 
the  final  Vessel  Radiated  Noise  ( VRN ) . 

It  is  not  intended  to  discuss  the  IIM  as  it 
is  a  ".tandard  microprocessor  system,  however  the 
operation  of  the  IFTM  and  the  use  of  the  signals 
to  produce  the  VRN  form  the  remainder  of  tnis 
paper. 

IFTM 

Capabi l lty 

The  module  is  capable  of  generating  a  total 
of  up  to  256  spectral  lines,  each  of  which  is 
ir.dependantiy  controllable  in  frequency,  phase 
and  amplitude.  The  frequency  resolution  with 
which  components  may  be  set  is  0.0125Hz  or 
better,  and  all  component  frequencies  are  derived 
from  a  common  crystal  controlled  oscillator. 
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FIG  1  PASSIVE  ACOUSTIC  TARGET  MODULE 


PASSIVE  ACOUSTIC  OUTPUT 


Since  the  module  Is  controlled  by  frequency  data 
it  is  possible  for  controlling  programs  to  take 
into  account  any  modification  to  the  spectrum  of 
the  radiatec  noise  which  may  result  from  propa¬ 
gation  througn  the  water.  The  use  of  frequency 
data,  rather  than  a  time /amplitude  sample  tech¬ 
nique,  allows  both  harmonically  related  and 
unrelated  components  to  be  generated  simultan¬ 
eously.  Each  spectral  line  is  independently 
controllable  in  frequency,  amplitude  and  phase 
but  may  be  set  to  be  a  harmonic  of  another 
component  if  required  by  setting  the  frequencies, 
as  close  as  possible  and  phase-locking.  This 
generation  technique  rather  than  the  amplitude 
sample  one  also  means  that  since  the  samples  are 
computed  by  the  module  as  required  In  real  time 
it  is  possible  to  ge"erate  a  very  low  frequency 
component  (e.g.  0,31 2  .Hi)  at  the  same  time  as 
high  frequencies  (e.g.  3KHz)  at  the  same  output. 


The  module  produces  four  analogue  output  wave¬ 
forms,  referred  to  as  W,  X,  Y,  Z. 

One  output  (W)  is  capable  of  producing  64 
components  up  to  a  frequency  of  2XHz .  If  th 
maximum  frequency  required  is  greater  than  2K.H* 
and  the  W  output  is  limited  to  32  components  the 
maximum  frequency  is  increased  to  4KHz .  Fig. 2 
shows  the  summary  of  the  capabilities  in  this 
format  and  this  format  is  assumed  for  the 
remainder  of  the  paper. 

An  overall  block  diagram  is  shown  in  Fig. 3. 
and  Figs. 4-7  show  more  detail. 
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Note 

Output 

w 

' '  y 

n~~  ^  n 

2,5 

Number  ot  components 

32 

64 

64 

64 

1,2 

Frequency  Range,  Hz 

U-4K 

U-1.6K 

0-400 

0-400  I 

2 

Frequency  Resolution,  Hz 

U.U125 

0.00625 

0.U015625 

0.U013625  | 

Frequency  Accuracy 

Crystal  controlled-bettei  than 

0.0i/. 

Amplitude  Control  Range 

>30  dB 

Amplitude  Control  increment 

<1.U2 

dB  to  -47  dB, 

i 

<2  dB 

to  -54  dB 

Phase  Control  Range,  deg. 

0  - 

360 

3 

Phase  Control  Increment,  deg. 

•  4 

“  1 

4 

Unwanted  sideband  level 

Better  than  -30  dB 

1 

with  respect  to  full 

tor  o/ p  frequency 

output 

< 1.5kHz. 

Better  than  - 

50  dB 

Better  than  -40  dB 

for  o/p  up  to 

4kHz 

J 

NOTES 

1.  The  module  is  capable  of  generating  trequencies  up  to  6.4  kHz,  3.2  kHz,  600  Hz  rind  HUD  Hz  res¬ 
pectively  at  outputs  W,X,Y,Z.  However,  above  the  ranges  quoted  the  wanted  signal  amplitude  tail 
and  the  unwanted  sideband  level  rises. 

2.  The  module  is  able  to  generate,  simultaneously,  up  to  the  stated  number  or  components  at  any 
integral  multiple  ot  the  stated  resolution,  within  the  stated  Frequency  Range. 

3.  Phase  control  accuracy  deteriorates  at  high  trequencies  and  is  subject  to  some  limitations. 

4.  The  principal  unwanted  sidebands  are  at  frequencies  13.1  kHz  +  Fw,  n.55  kHz  +  Fx,  1.6-*  kHz  +  Fv 
tor  wanted  trequencies  Fw,  Fx,  Fy,  F £  at  outputs  W,X,Y,Z  repectivelv. 

5.  Each  component,  may  be  either  a  sine  wave  or  a  square  wave.  In  the  latter  case  the  highest  har¬ 
monic  component  frequency  will  correspond  approximate ly  with  the  maximum  of  the  frequency  range 
quoted . 


Flu. 2  INVr-KSE  FOURIER  TRANSFORM  MODULE  CAPABILITY 
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FREQUENCE  PHA^E  AMPLITUDE 

DAI  A  DATA  OA’A 


Fig  3  INVERSE  FOURIER  TRANSFORM  MODULE- 
OVERAU  SYSTEM  BLOCK  DIAGRAM 


Realisation 

Overall  Description  (Fig. 3) 

Data  for  the  3*?  +  6-4  +  b4  +  64  components 
of  waveforms  W',X,Y,Z  respectively  are  loaded  into 
RAM's  under  program  control.  The  loading  is 
interleaved  with  reading  ot  data. 

At  fixed  (crystal  controlled)  intervals  the 
module  computes  amplitude  samples  for  the  four 
outputs  N,X,Y,Z.  Each  sample  comprises  the  sum 
of  the  samples  computed  for  each  ot  the  constit¬ 
uent  components,  which  are  computed  serially. 

The  computations  ot  samples  for  the  f out 
outputs  are  interleaved.  E<  ch  80  microseconds, 
approximately,  the  module  computes  a  full  sample 
for  Waveform  W  (’3  components),  i.alf  a  sample  (32 
components)  tor  Wveform  X,  and  one  quarter  of  a 
sample  fib  components)  for  Waveform  Y  or  Z.  Thus 
a  fulL  sample  of  W  is  computed  in  one  pass,  a 
full  sample  of  X  in  two  passes,  a  n!  i  ‘  ill  sample 
of  Y  and  Z  in  eight  passes.  Tie  computation  rate 
is  about  one  component  sample  per  microsecond. 

The  computation  ot  one  component  comprises: 

1 i )  Add  the  content  of  the  frequency 
store  to  the  previous  phase-angle  and  store  the 
new  data. 

(.  1  i )  Add  the  initial  phase  to  the  neu 
phase  angle, 

(iii)  Look  o  the  sine  ot  this  phase-ang le . 

(iv)  Multiply  the  sine  value  by  the 
mantissa  ot  tne  amplitude. 

(v)  bhitt  the  result  by  the  exponent  ot 
the  amplitude. 

(vi)  Add  the  result  to  the  contents  ot 
the  accumulator. 


Frequency  Control  (Fig. 4) 

Frequency  control  data  for  all  corapo  a 
~'s  entered  into  the  Frequency  Store  under  program 
control. 

On  each  pass,  data  for  each  component  is 
extracted  from  the  store  and  added  to  the  pre¬ 
vious  phase  angle  for  the  relevant.  component  from 
the  Phase  Angle  Store.  The  new  phase  angle  is 
then  output  via  the  Phase  Control  to  the  Sine 
Look-up,  and  the  new  phase  angle  replaces  the 
previous  one  in  the  Phase  Angle  Store. 

Thus,  on  each  pass,  the  Phase  Angle  (for  a 
given  component)  is  incremented  by  the  "Frequency”. 
The  scaling  is  so  chosen  that  a  Frequency  of  1 
increments  the  Phase  Angle  through  a  full  cycle 
in  2-U  passes,  which  takes  80  secs,  for  output  W, 
160  secs,  for  output  X  and  b40  secs,  for  outputs 
Y  and  Z.  The  corresponding  frequency  resolutions 
are  thus  0. 0125Hz,  0. 00625Hz  and  0. 0015625Hz. 

The  phase-angle  register  can  be  reset  to 
zero  by  a  control  from  the  Phase  Control  sub¬ 
system,  and  it  can  also  be  held  at  the  phase 
value  of  the  preceding  component  by  a  control 
from  the  Phase  control  sub-system. 
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CONTROL 


FROM 
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PHASE  CONTROL 


CONTROL 

Fig  U  FREQUENCY  CONTROL 


Phase  Control  (Fig. 5) 

Initia  i  ?ha«  angles  are  loaded  into  the 
Initial  Phase  store  under  program  control. 

On  each  pass,  the  Initial  Phase  for  each 
component  is  read  from  the  store  at  the  same 
time  as  frequency  data  is  added  to  the  phase 
angle  from  the  Frequency  Control  sub-system. 

I  ich  time  a  component  sample  is  computed 
the  sig.  of  the  phase  angle  is  examined,  and  if 
it  changt >  from  negative  to  positive  the  cycle 


count  for  that  component  is  inci  assented  -  The 
cycle  count  is  compared  with  the  stv.red  Harmonic 
Number  for  that  component  and  when  equivalence 
occurs,  and  if  a  marker  bit  (Mkl)  is  set  in  the 
control  data  to  indicate  that  the  component  is 
the  reference  for  a  harmonic  series  or  modulated 
wave,  the  control  logic  goes  about  resetting  the 
phase-angles  for  the  other  components  of  the 
group  as  they  are  computed.  The  other  components 
of  the  group  are  identified  by  another  marker  bit 
(Mk2T.  The  resetting  of  components  with  MK2  set 
continues  until  another  Mkl  is  seen.  (This  could 
be  the  same  Mkl  that  started  the  process,  seen  on 
the  next  pass). 


INITIAL  PHASE  OATA 


RESET  *0 
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Fig  5  PHASE  CONTROL 
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Fig  6  SINE  LOOK-UP  AND  AMPLITUDE  CONTROL 


Sine  Look-up  and  Amplitude  Control  (Fig. 6) 

The  most  significant  bit  of  the  phase-angle 
data  represents  180°,  and  is  used  as  the  sign 
bit.  The  next  bit  represents  90°  and  is  used  to 
complement  the  less  significant  bits  in  the 
second  and  fourth  quadrants.  The  sine  table 
represents  angles  from  0  to  90'. 

The  nine  value  from  the  look-up  table  is 
then  multiplied  by  the  mantissa  of  the  Amplitude 
data.  The  result  is  shifted  left  (to  positions 
of  greater  significance)  by  the  exponent  of  the 
Amplitude  lata. 

The  data  is  then  negated  if  the  sign  bit  is 

set. 

1*  a  marker  bit  is  set  in  the  control  data, 
the  sine  table  is  disabled  and  gives  an  output  of 
1.  Thus  a  square  wave  (+1)  is  generated  for  any 
component  which  has  this  marker  bit  set. 


Accumulator  and  D/A  Conversion  (Fig. 7) 

Components  of  a  waveform  sample  are  added 
in  the  accumulator.  The  accumulator  register  is 
first  cleared  and  successive  components  are  then 
added  to  the  register  contents.  When  the  sample 
is  complete  (i.e.  all  components  have  been  added) 
the  accumulator  register  content  is  transferied 
to  the  D/A  Register  and  the  accumulator  cleared 
again. 

Multiplexing 

The  system  as  outlined  is  multiplexed 
between  four  outputs,  W,  X,  Y  and  Z. 

On  each  pass  (80  S)  32  components  of  W,  32 
components  of  X  and  16  components  of  either  Y  or 
Z  are  computed  and  separately  accumulated.  A 
complete  sample  (32  components)  of  W  is  thus 
computed  on  every  pass,  two  passes  are  required 
for  a  full  sample  of  X,  and  8  passes  are  used  to 
accumulate  full  samples  of  Y  and  Z.  When  the 
complete  samples  have  been  accumulated  they  are 
transferred  to  the  appropriate  D/A  register  and 
the  accumulators  reset. 
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Fig  7  ACCUMULATOR  ANO  O/A  CONVERSION 
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PASSIVE  ACOUSTIC  OUTPUT  1 


PASSIVE  ACOUSTIC  OUTPUT  2 


FIG  8  DOUBLE  PASSIVE  ACOUSTIC  MODULE 


Utilisation  of  Generated  Waveforms 

Fig. 2  shows  the  utilisation  of  the  gen¬ 
erated  waveforms  to  produce  the  composite  VRN. 

The  W  and  X  outputs  form  the  discrete  frequencies 
generated  by  the  target.  The  Y  and  Z  outputs  are 
used  to  modulate  suitably  filtered  noise  to 
represent  cavitation  and  machinery  noises. 
Unmodulated  flew  noise  is  also  produced  in  the 
module.  These  components  are  then  added  together 
in  the  correct  ratio  by  use  of  attenuators  and  a 
summing  amplifier  to  form  the  complete  VRN  for 
use  in  the  simulators. 

Optional  Alternative  Generated  W  .veforms 

Severs1  options  are  available  for  the  1FTM 
but  tne  major  use  is  to  divide  the  Y  and  Z 
outputs  into  two  separate  outputs  (Yl,  Y2,  Z1  and 
Z2)  each  with  32  components  available.  Hence  It 
it  possible  to  configure  the  module  to  produce 
two  target  VRNs  with  W  and  X  providing  the 
discrete  components  for  each  target  and  with  Yl 
and  Y2,  Z1  and  Z2  fonning  the  modulating  Signals. 
This  obviously  halves  the  number  of  lines  avail¬ 
able  to  each  target  but  nevertheless  produces  a 
good  target  model.  The  modulating  signals 
obviously  have  to  modulate  separate  noise  sources 
and  separate  flow  noise  is  also  required.  This 
configuration  is  shown  in  Fig. 8. 
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ABSTRACT 

""Most  anti-submarine  warfare  (ASW)  trainers  for  either  airborne  or  sea¬ 
borne  platforms  require  the  simulation  of  ocean  acoustic  transmission  loss  to 
model  the  submarine  signals  received  by  sonobuoys  or  towed/hull  arrays. 
Current  trainer  software  usually  employs  transmission  loss  calculations 
based  on  acoustic  theories  which  assume  the  ocean  environment  to  be  range- 
independent.  That  is,  these  theories  assume  that  such  variables  as  water 
depth  and  the  sound  speed  profile  (ssp)  do  not  change  along  the  path  of  acous¬ 
tic  transmission.  Although  such  ocean  modeling  is  not  realistic,  the  final 
calculations  have  often  been  acceptable  due  to  their  high  predictability. 

With  operational  fleet  equipment  gaining  in  sophistication,  the  ocean 
acoustic  software  which  stimulates  this  equipment  must  be  refined.  The 
introduction  of  range-dependent  transmission  loss  calculations  is  one  of  the 
most  important  refinements.  However,  the  associated  theoretical  problems 
are  complex,  and  their  solutions  are  difficult  to  implement  practically, 
especially  in  simulators  that  require  real-time  software. 


This  paper  discusses  some  of  the  complications  tha.  range-dependent 
ocean  acoustic  modeling  introduces  and  uses  a  prototype  model,  the  Multiple- 
Profile  Configuration  (MPC)  Ocean  Model,  developed  by  the  Link  Simulation 
Systems  Division  of  the  Singer  Company,  to  present  specific  methods  of  resolv¬ 
ing  these  compl icat i ons^ 


The  MPC  Ocean  Model  includes  both  off-line  and  on-line  modules.  The 
former  pre-calculates  those  databases  that  depend  only  on  tha  gaming  area  to 
be  used,  and  the  latter  calculates,  in  real-time,  the  transmission  loss  of 
specific  range-dependent  acoustic  paths.  The  on-line  module  includes  such 
features  as  selective  ray  tracing  for  deep  ocean  calculations,  normal-mode 
calculations  for  shallow  ocean  areas,  and  simulation  of  seamount  shadowing. 
All  are  used  within  the  framework  of  range-dependent  sound  speed,  water  depth, 
and  bottom  type.  This  model  is  presently  incorporated  into  a  P3-C  Orion, 
Weapon  System  Trainer. 


INTRODUCTION 


Tactical  considerations  of  anti¬ 
submarine  warfare  (ASW)  lead  to  a  variety 
of  complicated  operational  scenarios.  A 
simplified  axampie  is  depicted  in  Figure 
1,  where  emphasis  is  placed  on  the  related 
ocean  acoustics.  A  jiven  scenario  may 
involve  many  airborne  and  seaborne  plat¬ 
forms  trying  to  locate,  evade,  or  attack 
each  other.  Applications  of  underwater 
acoustics  play  a  major  role  in  *-he  first 
two  of  these  objectives.  Consequently, 
ASW  trainers  must  simulate  ocean  acoustic 
phenomena,  and  of  course,  they  must  do  so 
in  real-time  to  be  effective.  Thus,  in 
addition  to  various  non-acoustic  systems 
(e.g.  radar,  position  keeping,  magnetic 
anomaly  detection),  such  traim  rs  also 
include  many  software/hardware  systems  in 
order  to  simulate  the  relevant  acoustics: 
signal  generation,  transmission,  recep¬ 
tion,  analysis,  and  display. 


Figure  1.  ASW  Ocean-Acoust i e  Scenario. 

Tli  is  paper  focuses  on  the  real-time 
software  mode  li  rig  of  ocean  acoustic 
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transmission.  The  prototype  for  this 
discussion  is  the  Multiple-Profile  Con¬ 
figuration  ( MPC )  Ocean  Model  developed 
by  the  Link  Simulation  Systems  Division 
(Link  SSD)  of  the  Singer  Company,  and 
which  is  presently  incorporated  into  a 
P3-C  Orion  aircraft  trainer.  The  MPC 
Ocean  Model  is  a  software  package  which 
simulates  the  acoustic  transrr  salon  loss 
for  target  signals  received  oy  passive, 
omnidirectional  sonobuoy  systems.  (This 
calculated  loss  is  input  to  a  digital 
signal  generator;  appropriate  target 
characteristics  are  incorporated,  and  the 
output  stimulates  the  on-board  acous  tjc 
data  processor  providing  the  displays  at 
the  aircraft  sensor  stations.) 

The  MPC  Ocean  Model  has  as  its  design 
basis  a  combination  of  U.S.  Navy  research 
and  development  (R&D)  transmission  loss 
models,  a  feature  it  shares  with  most 
other  trainer-based  models.  The  reason  is 
twofold.  First,  the  major  customer  for 
ASW  trainers  is  the  Navy,  whose  trainees 
and  fleet  operators  are  briefed  in  the 
areas  of  acoustics  using  calcula*-  i  ons  made 
by  Navy  research  models.  Second,  the  Navy 
scientific  laboratories  have  spent  forty 
years  performing  basic  research  and  gene¬ 
rating  most  of  the  consequent  computer 
models.  They  have  tested  and  refined 
fcheir  theories  by  numerous  at-sea  experi¬ 
ments.  Due  to  the  unwieldy  ocean  environ¬ 
ment  and  the  large  number  of  uncontrolla¬ 
ble  variables,  the  theory-experiment 
cycling  process  has  been  slow  and  expen¬ 
sive.  Thus,  it  is  cost-effective  for 
trainer-configured  models  to  utilize  the 
fruits  of  this  labor.  It  must  be  remem¬ 
bered,  however,  that  whereas  the  various 
research,  models  may  execute  for  several 
seconds  or  minutes  to  oh  ain  one  calcu¬ 
lated  result,  their  trainer-based  counter¬ 
parts  must  run  in  real-time,  which  is  typ¬ 
ically  less  than  one  second.  This  engi¬ 
neering  problem  is  overcome  by  using  vari¬ 
ous  approximations  to  the  full  theories, 
by  employing  table  look-ups  of  pre¬ 
calculated  databat.es,  or  by  combinations 
of  t';,->se  approaches. 

The  past  decade  has  seen  ocean  acous¬ 
tic  research  modeling  rise  to  a  relatively 
high  level  of  ‘ heoretical  and  computa¬ 
tional  sophistication.  Similar  advances 
are  being  realized  in  the  software/ 
hardware  of  the  newest  on-board  Navy  fleet 
operational  systems.  Hence,  the  detailed 
effects  of  the  ocean  environment  on  acous¬ 
tic  transmission  re  now  better  understood 
and  are  also  becoming  more  recognizable. 
This  presents  a  rather  complex  problem  in 
the  design  of  realistic  ASW  simulation 
systems,  because  their  level  of  sophisti¬ 
cation  must  parallel  the  fleet  systems. 
To  get  enhanced  transmission  loss  calcula- 
t  i  ons  in  real-time,  considerable  computa¬ 
tion  power  is  required.  As  an  example, 
out  of  four  computers  comprising  the 
above-mentioned  PJ-C  traine-,  on*7*  computer 
is  entirely  dedicated  for  the  MPC  Ocean 
Model.  This  is  notable  because  transmis¬ 
sion  loss  is  only  part  of  the  acoustics 


picture,  and  there  are  also  many  non- 
acoustic  systems  to  be  simulated.  On  pre¬ 
vious  ASW  trainers,  much  less  sophisti¬ 
cated  transmission  loss  models  have 
occupied  substantially  smaller  portions  of 
similar  computers. 

PREVIOUS  DESIGN  APPROACHES 

Perhaps  the  best  known  R&D  transmis¬ 
sion  loss  model  is  the  Fast  Asymptotic 

Coherent  Transini  ss i  on  (FACT)  model,  (1'2) 
further  distinguished  by  its  use  at  Fleet 
Numerical  Oceanography  Center  (FNOC)  to 
generate  standard  aircrew  ASW  briefing 
packages.  These  Acoustic  Sensor  Range 
Prediction  (ASRAP)  packages  are  used  by 
ASW  instructors/trainees  as  well  as  the 
fleet.  Each  package  includes  plots  of 
transmission  loss  versus  range  for  various 
frequencies  and  sensor  depth  combinations. 
For  example,  for  the  four  standard  fre¬ 
quencies  (50,  300,  850,  1700  Hz)  and  four 

combinations  of  t arget /sonobuoy  depths 
(each  being  "shallow"  or  "deep"),  the 
ASRAP  package  for  a  given  ocean  area 
includes  sixteen  transm.i  si  son  loss  curves. 
These  curves  sufficiently  cover  the 
FACT-generat ed  loss  data,  for  FACT  treats 
the  qiven  ocean  area  as  being  environ¬ 
mentally  range-independent.  This  means 
that  the  water  depth  is  fixed  ( flat -bot  torn 
ocean  model)  and  the  sound  speed  profile 
and  Lot  tom  type  remain  constant  with 
range . 

With  such  precedence,  i*  i s  no  wonder 
that  the  FACT  model  figures  heavily  in 
the  design  bases  of  most  previous  ASW 
simulators.  In  particular,  P 3 -type 
simulators  have,  for  10  years,  employed 
a  simple  table  look-up  approach.  The 
on-line  model  linearly  interpolates/ 
extrapolates  the  pre-ca lculated  FACT  loss 
data  for  the  actual  frequencies,  ranges, 
an  5  depths  of  interest.  The  resultant 
losses  are  not  necessarily  correct,  but 
they  provide  the  advantage  of  being  very 
accurately  predictable,  with  smooth 
transitions  between  ASRAP  values.  Link 
SSD-deve l oped  submarine  trainers  of 
younger  vintage  employ  FACT  directly 
on-line,  although  modified  and  re-coded 
tor  reai-time.  This  approach  signifi¬ 
cantly  increases  the  required  comparer 
power,  but  it  produces  agreement  with  FACT 
at  all  frequencies,  ranges,  and  depths. 

Other  traine  a  (e.g.  the  S3 -A  Viking 
aircraft  series)  also  use  a  direct  calcu¬ 
lation  approach.  A  simplified  ocean 
environment  is  modeled,  using  standard 
R&D-bused  techniques  approximated  »o  yield 
real-time  transmission  loss  calculations. 
The  classically  observed  effects  are 
modeled,  sometimes  better  than  FACT,  but 
with  the  disadvantage  of  to*  'natch :  ng  that 
mode  1 . 


What  all  of  these  approacr.es  have  in 


common  are  r he i r 
dependence  in  the 
Consequently,  their 
able,  but  they  are 


exclusion  or  r  a  n g  e 
ocean  r*  n  v  i  r  o  n  m  e  n  '  . 
results  ar<  predict  - 
often  inadequate  for 
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modeling  a  realistic  ocear  area.  However, 
with  the  mutual  increase  in  computer  power 
and  research  model  capabilities,  range 
dependence  can  now  be  incorporated  into 
the  simulation  computer  complex,  as  it 
should  be  with  fleet  equipment  gaining  in 
sopni st i cat  ion. 

RANGE-DEPENDENT  TRANSMISSION  LOSS 
Background 

The  ocean  environment  is  nonhomogen- 
eous  in  every  respect.  Although  the 
parameters  that  vary  with  range  are  many, 
•those  parameters  having  an  appreciable 
effect  on  acoustic  transmission  loss  have 
been  identified  and  studied  by  numerous 
scientists.  These  parameters  are  illus¬ 
trated  in  Figure  2.  The  short -dashed 
lines  shown  emanating  from  the  target  in 
the  figure  are  acoustic  rays.  Ray  tracing 
(about  which  more  wall  be  said  later)  is 
one  method,  fortunately  a  very  graphic 
one,  of  analyzing  how  emitted  signals  are 
transmitted  to  an  underwater  receiver.  It 
is  used  by  a  number  of  research  models 
including  FACT,  and  is  the  basis  for  the 
deep  ocean  calculations  of  the  MPC  Ocean 
Model.  Ray  tracing  is  easy  to  extend 
theoretically  to  include  a  range-dependent 
environment;  this  has  been  accomplished 
for  example,  by  the  following  research 
models:  RAYWAVE  II  [Naval  Ocean  Systems 

Center  (NOSC)]/3)  TR1MAIN  [Naval  Research 
Laboratory  (NRL)l,^4^  and  GRASS:  Germi¬ 

nating  Ray  Acoustics  Simulation  System 

(NRL).^3^  To  implement  range-dependent 
ray  tracing  practically,  however,  is 
difficult,  especially  in  simulators  that 
require  real-time  software. 


Figure  2  .  Ra  n  g  e 
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intercepted  ray.  Also,  note  how  the 
direction  of  energy/  transmission  changes 
after  an  encounter  ./'wi th  the  ocean  bottom. 
Ray  tracing  assume^  that  incident  rays  are 
specularly  refle/cted  off  of  the  ocean 
bottom.  Hence, / a  change  in  the  bottom 
slope  yields  an'  effective  change  in  the 
incident  grazing  angle,  thus  producing  a 
reflected  ray/ that  is  depende  it  on  the. 
bottom  slope  (/see  inset  in  Figure  2).  The 
effect  that  a/  sloping  bottom  versus  a  flat 
bottom  may  >iave  on  transmission  loss  is 
dramat  i  ca  1 1/y  illustrated  in  ligure  3, 
which  was  /generated  by  the  GRASS  model. 


Figure  3.  Effect  of  Sloping  Bottom 
on  Transmission  Loss. 

Of  course,  some  of  the  Lot t om-bounce 
ray  energy  is  transmitted  into  the  bottom 
and  lost,  a  phenomenon  which  is  also 
dependent  on  *  le  effective  grazing  angle. 
This  bottom  loss  depends  also  on  the 
type  of  material  comprising  the  bottom. 
T ! I  e  U . b .  S3 v  5  »  0 c  e  a  i i o g  r  a  p  h  >  q  Office 

(NAVOCEANO)  has  categorized  nine  standard 
bottom  types  (  me  to  nine,  ranging  from 
rock  to  light  mull.  Their  effects  have 
been  empirically  ae*  e ruined y  standard 

bottom  loss  curves^)  are  shown  in 
Figure  4. 

Often,  the  most  important  range- 
dependent  parameter  is  the  sound  speed 
profile  (ssp>.  Its  effect  is  manifested 
in  ray  tracing  by  a  change  in  the  curva¬ 
ture  of  t  ht  rays  as  they  are  traced. 
Figure  2  shows  a  frontal  zone  across  which 
the  ssp  varies  from  sspl  on  the  target 
side  t  )  ssp2  on  the  sonobuoy  side.  The 
effect  that  such  multiple  profiles  may 
have  or.  ran  smissi  on  loss  is  shown  i  n 
Figure  5  (also  generated  by  GRASS).  Tie 
change's  in  the  convergence  zones  (cz's) 
tre  notably  conspicuous. 
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Figure  4.  Nominal  Bottom  Loss  Curves  for 
the  Frequency  Range  1-4  kHz,  Divided  into 
the  nine  NAVOCEANO  Classes. 
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Figure  5.  Effect  of  Range-Dependent  Sound 
Speed  Profile  (ssp)  on  Transmission  Loss. 

The  limiting  case  reached  by  a 
steeply  sloping  bottom  is  the  so-called 
shallow  ocean.  Such  an  area  corresponds 
to  an  ocean  region  overlying  the  conti¬ 
nental  shelf.  In  a  shallow  ocean  environ¬ 
ment,  the  surface  and  bottom  interactions 
of  the  acoustic  energy  are  so  qreat  as  to 
render  ray-tracing  techniques  ineffective, 
and  an  entirely  different  theory  must  be 
used,  whereby  the  ocean  becomes  an  acous¬ 
tic  waveguide  (see  discussion  below).  An 
additional  complication  is  the  amplified 
importance  of  the  unconsolidated  sediment 
overlying  the  bottom.  The  type  of  sedi¬ 
mentary  material  and  it*  thickness  both 


become  very  influential  on  the  transmis¬ 
sion  loss  curves.  Rather  than  present 
another  loss  versus  range  comparison, 
Figure  6  shows  the  effect  that  different 
shallow  ocean  sediment  types  may  have 
on  the  beam  response  of  a  horizontal 
array  receive^  (The  calculations  were 
made  using  an  unshaded,  linearly-spaced 
discrete  array  with  linear  steering,  using 

normal-mode  theory^7  ^  .  )  Whereas  the 
response  pattern  for  the  sand-silt-clay 
environment  is  nearly  plane  wave  in 
character,  a  change  to  coarse  sand  sedi¬ 
ment  (all  other  parameters  being  equal) 
severely  degrades  the  array  response. 
Beam  sh'ftjng,  broadening,  and  splitting 
are  all  evident.  Although  our  prototype 
P3-C  is  not  required  to  model  array  re¬ 
ceivers,  the  illustration  in  Figure  6  is 
useful  for  surface  ship  trainer  design 
which  must  simulate  towed  arrays. 


SAND  — S'LT— CLAY  COARSE  SAND 


Figure  6.  Beam  Response  vs.  Steering  De¬ 
lay  Time.  A  100-element  array  was  placed 
at  9  km  from  the  source  in  a  downward 
refracting  ssp.  The  calculations  shown 
are  for  (a)  sand-silt-clay  and  (b)  coarse 
sar.d  sediment  types.  Tick-marks  along 
the  y-axis  are  spaced  at  5  dB  intervals'. 

Imp  ientation 

In  order  to  include  range  dependence 
in  the  modeled  ocean  environment,  i t 
is  clear  that  the  FACT  model  must  be 
abandoned  (or  extensively  modified). 
Viable  alternatives  include  many  Navy- 
developed  models  such  as  those  mentioned 
above  ( RAYWAVE  II,  TRIMAIN,  GRASS)  and  the 
following:  FACT EX  (FACT  Extended  to 

include  range  dependence),  at  Naval 
Ocean  Research  and  Development  Activity 

(N0RDA);  RAYMODE  X  (or  a  range-dependent 
update  of  it),  at  Naval  Under vater  Systems 

Center  (NUSC);(8'9)  the  FFP  (Fast  Field 

Program),  at  NUSC;(10)  the  PE  (Parabolic 
Equation)  Program,  at  N0RDA  and 

NRL;  kkl,12)  RP-70,  at  FN0C;#  and  M0A7L 

*  No  documentation  currently  available. 
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Abandoned  along  with  the  direct  use- 
of  FACT,  must  be  the  simplistic  table 
loo<-up  approach.  As  we  have  seen,  range 
dependence  introduces  a  host  of  new 
variables  to  be  considered  for  a  given 
ocean  gaming  area  (typically  300  nm  to  500 
nm  square).  Along  a  given  transect 
. t a r g e t - to- sonobuoy  path,  as  viewed 
top-side),  there  may  be  multiple  bottom 
slopes  and  bottom  types  and  multiple  ssp 
variations.  Because  the  gaming  area  is 
three-dimensional  [as  opposed  to  the 
two-dimensional  (r,z  only)  environment  of 
the  research  models],  changes  in  lateral 
(x,y)  locations  of  the  sensors  produce  an 
almost  infinite  variety  of  important  range 
dependencies.  (Refer  to  Figure  2  for 
definitions  of  the  axes/dimensions.)  For 
example,  i.  n  Figure  2,  j.  f  the  target  j.  s 
moved  perpendicularly  away  from  the  plane 
of  the  page,  then  the  new  transect  crosses 
a  wider  frontal  zone  and  the  bottom  slopes 
are  less  pronounced.  The  pinnacle  shown 
(or  others)  may  or  may  not  intersect  the 
transect.  It  is  easy  to  determine  that, 
transmission  loss  table  sizes  for  a 
look-up  approach,  can  exceed  100  million 
values.  Data  storage  and  access  time  both 
become  problems.  Further,  i mpl ement ation 
of  interpolations  and  corrections  to  table 
look-ups  is  not  straightforward.  More 
fundamentally,  j.t  is  not  theoretically 
clear  what  these  corrections  should  be. 
For  example,  if  a  particular  bottom  slope 
v  a  r  i.  ation  along  a  given  transect  is 
changed  or  moved  out  in  range,  then  the 
rays  traced  may  or  may  not  be  affected. 
Whether,  and  how,  are  dependent  on  a 
number  of  variables  in  a  way  not  readily 
determined  without  actually  tracing  the 
rays. 


The  straightforward  solution  to  these 
problems  i. s  to  calculate  the  transmission 
loss  on-line,  using  the  actual  range- 
dependent  ocean  conditions  along  the 
transect.  (This  is  the  approach  of  the 
MPC  Ocean  Model,  as  discussed  below.) 
To  obtain  real-time  results,  many  approxi- 
inations  to  the  theory  must  be  made. 
These  must  be  carefully  chosen  to  preserve” 
both  the  desired  effects  and  their  theo¬ 
retical  authenticity. 


The  final  thing  abandoned  is  strict 
agreement  with  FACT,  or  for  that  matter, 
with  any  research]  model  or  any  set  of  ex¬ 
perimental  data.  The  advantage  gained  by 
this  sacrifice  is  the  real-t i me  '  modeling 


5i  th  Jr£  ,  ,  range-dependent  effect.  . 
*  ‘  "  and  trouble,  however, 

predictability  can  be  regained.  The 
trainer-based  model  can  be  exercised  in  an 
off-line  research-oriented  environment  to 
generate  loss  and  even  ray-trace  plots. 
These,  in  turn,  can  be  distilled  to  a 
reasonable-size  package  for  use  in  brief¬ 
ing  trainees  and  by  instructors  during  the 
training  mission. 


THE  MPC  OCEAN  MODEL 
Design  Approach 


The  MPl  Ocean  Model  has  a  modular 
structure  which,  to  allow  for  modifica¬ 
tions  and  additions,  follows  the  principle 
of  functional  decomposition.  It  consists 
of  both  off-line  and  on-line  portions,  as 
shown  in  Figure  7.  Off-line  functions 
include  gaming  area  specification,  calcu¬ 
lation  of  gaming  area  grid  tables,  and 
other  calculations  of  data  that  do  not 
depend  on  specific  transects.  These 
calculations  are  performed  to  save  time 
on-line,  and  result  in  a  database  of 
approximately  fifty  thousand  words  per 
gaming  area  (about,  ninety  percent  of  this 
is  for  the  shallow  ocean).  The  on-line 
modules  calculate  the  transmission  loss  at 
the  four  passive  ASRAP  frequencies  for  up 
to  51  different  target-sonobuoy  pairs  in 
about  ten  seconds.  Although,  the  average 
time  per  transect  is  roughly  two  hundred 
milliseconds,  the  actual  time  for  any 
given  transect  can  vary  from  several 
milliseconds  to  about  a  second.  For  this 
reason,  most  of  the  modules  execute  in 
background.  They  are  called  on  demand  by 
the  fixed-frame  Target-Sonobuoy  Scheduler, 
which  tracks  the  time  between  updates  and 
assigns  priority  to  each  target-sonobuoy 
pair.  The  Geometry  Model  calculates  all 
the  non-acoustic  parameters  along  the 
scheduled  transect  and  then  calls  the 
Shallow  Ocean  or  Deep  Ocean  Model.  These 
two  transmission  loss  models  are  discussed 
below . 


►  -  -mckmouwo 


Figure  7.  Software  Work  Breakdown 
Structure  of  the  MPC  Ocean  Model. 
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Even  with  a  whole  computer  dedicated 
for  the  MPC  Ocean  Model,  real-time  calcu¬ 
lations  arc  net  easy  *-o  achieve.  To  this 
end,  all  of  the  on-line  modules  are 
written  in  Assembler  language.  Further¬ 
more,  they  employ  mostly  single-precision 
(16-bit)  words  in  carefully-scaled  fixed- 
point  arithmetic.  Double  precision  is 
occasionally  used  when  needed,  as  is  the 
floating-point  processor.  With  computers 
gaining  in  speed,  use  of  higher-level  lan¬ 
guages  (e.g.,  FORTRAN)  and  all  floating¬ 
point  arithmetic  nay  hp  possible  in 
similar  efforts  of  the  future,  but  careful 
deliberation  is  warranted-  The  abilities 
of  available  compilers  to  generate  effi¬ 
cient  machine  code  must  be  scrutinized. 

As  stated  before,  approximations  tc 
the  full  theories  are  also  required;  some 
of  these  are  discussed  below. 

MPC  Deep  Ocean 


Sped f i cat  j  on .  The  MPC  Deep  Ocean 
Model  calculates  the  transmission  loss  at 
the  four  AS RAP  frequencies  when  the 
selected  transect  is  in  water  deeper  than 
150  meters.  A  deep  ocean  gaming  area  is 
shown  in  Figure  8,  where  range  dependence 
in  water  depth  is  signified  by  the  bottom 
contour  lines.  Figure  8a  was  traced  from 
a  bathymetric  chart  of  an  area  southeast 
of  Japan.  Figure  8b  illustrates  an 
acceptable  specification  of  this  same  area 
for  the  MPC  Ocean  Model.  The  circles 
represent  pinnacles  with  top-depths  as 
marked.  There  are  many  sloping-bottom 
areas,  for  example,  the  one  marked  "A”. 
An  exemplary  flat-bottom  area  is  marked 
"B’  .  A  9000-meter  trench  is  marked  "C". 
Although  not  illustrated,  ridges  are 
modeled  similarly. 

For  t ne  same  gaming  area  as  shown 
in  Fiqure  8,  specification  of  a  range- 
dependent  bottom  type  is  shown  in  Figure 
9.  The  actual  bottom  variations  are 
given  in  Figure  9a.  Fiqure  9b  illustrates 
the  rectangu 1 ar-regi on  input  method  used 
by  the  model.  The  minimum  resolution 
allowed  by  tp-?  MFC  Ocean  Model  is  8  nm. 
Co r r e s pond i n q 1 y ,  the  off-line  module 
generates  a  38x38-value  grid  <  able  for 
later  access  on-line. 

In  Figure  10,  the  ocean  currents 
and  a  front  region  are  shown.  Currents 
are  not  used  directly  in  the  transmission 
loss  calculations.  They  merely  account 
for  the  dynamic  drift  of  tarqets  and 
sonobuoys,  and  are  stored  in  a  grid 
••able  similar  to  that  for  bottom  types. 
However,  associated  with  currents  are 
temperature  fluctuations,  which  can 
give  rise  to  a  front,  across  which  the 
ssp  showi,  its  greatest  variations.  As 
indicated  in  Figure  10b,  the  MPC  Ocean 
Model  allows  for  regions  of  constant 
ssp  separated  by  frontal  zones  across 
which  the  ssp  varies.  Some  of  the  front  - 
dependent  data  are  generated  off-line 
to  save  time.  Some  are  not,  however, 
because  the  r^nge-var i a? i on  across  a  given 
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Figure  8.  Gaming  Area.  A  300-nm  square 
area  showing  bottom  contours  (depth  unit* 
arc  meters).  (a)  Actual  ar*a.  (b)  Saae 
area  a«  simulated  by  the  Mf*C  Ocean  Model. 
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Figure  **  .  l(of  f  om  Type.  Specification 
nsinq  Mk*  N  WOCEXNO  numbering  system, 
(a)  /\c*  Jal  area.  (b)  Simulated  area. 
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Figure  10.  Front-Region  Spec i * i cat j on . 
(a)  Actual  area,  shoving  currents  that 
help  produce  the  front.  (b)  Simulated 
fror.t,  demonstrating  that  the  variation 
from  Profile  1  to  11  dej'ends  on  how  the 
front  is  crossed. 


(  b )  S i mu  1  at  ed 
t  he  v *r l at  ion 


front  will  change  with  respect  to  the 
bearing  of  a  specific  transect  (compare 
"A"  to  "B"  in  Figure  10b). 

It  should  be  pointed  out  that  the 
gaming  area  of  Figures  8-10  illustrates 
the  level  of  range  dependence  afforued  by 
the  MPC  Ocean  Model,  and  is  not  intended 
for  the  no /ice  trainee.  The  simulator- 
based  training  period  must  be  organized 
(and  perhaps  extended)  so  t0  progress 
from  the  simpler  to  the  more  complex  ocean 
envi ronment  s . 

Caicmar ions.  The  deep  ocean  trans- 
mi  ssi  on  less  calculations  are  based 
primarily  on  ray-trace  theory,  which 

arises  as  follows:  theoretically, 

the  transmission  of  acoustic  energy  obeys 
the  wave  equation,  which  for  a  harmonic 
point  source  at  depth  z^  takes  the  form: 

£.'2  +  (;)2]  :  =  -  '(x)  *(y)  'U-zQ)  •  U) 

Here,  =  2  f  is  the  angular  frequency. 
The  sound  speed  c  and  the  velocity  poten¬ 
tial  I  =  e  3  '  f  are  both  functions  of 

position.  The  acoustic  pressure,  hence 
the  transmission  loss,  can  be  calculated 
from  the  velocity  potential.  Assumina  a 
solution  in  the  form 

j  ^ 

;■  =  Ae  ( A , W  functions  of  position)  (2) 

results  in  the  eikonal  equation. 


(J£)2  +  (-|)2  + 


(3) 


which  is  the  basis  for  ray  tracing.  The 
surfaces  of  constant  phase  (W  =  constant) 
are  the  wave  fronts,  and  the  norma,  s  to 
these  are  acoustic  rays. 


made  by  many  research  models,  with  the 
convenient  result  that  the  ray  path  in 
each  layer  follows  the  arc  of  a  circle. 
This  is  an  aid  to  real-time  calculations, 
whereby  several  trigonometric  table  look¬ 
ups  (along  with  a  few  other  calculations 
and  extensive  logic)  suffice  to  trace  a 
given  ray  through  a  given  layer.  For  some 
rays,  such  as  the  one  labeled  "A"  in 
Figure  11,  symmetry  allows  the  stored  data 
for  several  layers  to  be  quickly  "folded 
out"  in  rar.ge  (without  recalculation).  At 
the  onset  of  the  front,  however,  new 
calculations  must  be  made.  After  studying 
several  research  models  (including  GRASS 
and  TRIMAIN),  the  front-region  techniques 
of  RAYWAVE  II  were  adapted  for  use  in  the 

MPC  Ocean  Model. ^ ^  As  shown  in  Figure 
11,  the  front  js  broken  into  triangular 
regions.  For  each  triangle,  an  average 
sound  speed  gradient  is  determined,  which 
now  has  a  range-component  as  well  as  a 
depth-component.  By  a  rotation  of  coor¬ 
dinates,  however,  the  ranoe  dependence  is 
temporarily  "eliminated";  the  ray  is  effi¬ 
ciently  traced  to  the  next  boundary  in  the 
standard  manner,  and  then  an  inverse  coor¬ 
dinate  rotation  is  performed.  Although 
quick,  it  is  found  that  the  front -trace 
calculations  consume  the  major  portion  of 
the  total  transect  calculation  time.  This 
is  the  price  paid  for  range  dependence. 

To  model  a  sloping  bottom,  the  range 
and  grazing  anqle  of  the  ray  at  the  point 
of  bottom  interception  must  be  determined 
(refer  to  Figure  2).  This  requires 
finding  a  solution  for  the  intersection 
of  two  equations,  one  Dei ng  the  arc 
of  a  circle  (the  ray),  and  the  other  a 
sloping  line  (the  bottom).  To  save  time 
during  most  of  the  "normal"  trace,  these 
considerations  (i.e.,  calculations  for 
determination  of  a  bottom  intercept)  are 
not  made  until  a  ray  has  penetrated  the 
so-called  "false  bottom,"  a  pre-ca leu lated 
parameter  which  is  illustrated  in  Fiqure 
11  (ray  penetration  occurs  at  point  "B"). 


The  calculat ional  procedure  js  to 
trace  the  various  rays  from  the  source  out 
to  the  receiver  range,  assign  the  rays  to 
families  (having  like  trace  histories), 
and  then  perform  standard  ra> -i nt ens i t y 
spreading  t  eclmiques  to  get  the  trans¬ 
mission  loss.  To  accomplish  this  in  real¬ 
time,  the  MPC  Ocean  Model  makes  a  funda¬ 
mental  approximation.  Only  20  rays  are 
traced,  as  opposed  to  hundreds  or  even  a 
thousand  as  are  traced  by  most  research 
me  dels.  To  enhance  the  value  of  these  20 
rays,  there  is  extensive  logic  to  direct 
the  ray  fan  up  or  down,  and  to  adjust  the 
angular  spacing  between  the  individual 
rays  leaving  th*.  source. 

Equation  (j>  shows  the  dependence  of 
ray  tracing  on  variations  in  sound  sjxred. 
Consider  first  a  constant  ssp  (non-front) 
region,  such  as  that  where  the  targe*  i a 
located  in  Fiqure  11.  As  shown  there,  the 
MPC  Ocean  Model  assumes  a  pieco-wisi* 
linear  ssp,  with  up  t0  five  layers,  in 
each  of  which  the  sound,  speed  gradient 
is  constant.  This  is  an  assumption  also 


Fiqure  11.  Bouniarien  Between  Hay-Trace 
Unions . 
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Having  extended,  extolled,  and 
exhausted  ray- tracing  techniques,  it  must 
be  conceded  that  there  are  special  cases 
where  the  theory  breaks  down.  In  qoinq 
from  Equation  (1)  to  Equation  (3),  one 
basic  assumption  must  be  made.  The 
fractional  change  in  the  sound  speed 
gradient  over  the  distance  of  an  acous¬ 
tical  wavelength  must  be  small  in  compar¬ 
ison  with  the  gradient.  in  terms  of 
frequency,  this  means  that: 


Thus,  at  the  bottom  of  the  ocean,  the  ssp 
discontinuities  are  such  as  to  render 
Equation  (3),  hence  ray  tracing,  invalid. 
In  the  deep  ocean,  however,  where  there 
are  few-  bottom  encounters  per  ray,  bottom 
loss  may  be  assigned  in  an  empirical 
manner  (the  curves  were  given  in  Figure  4) 
and  the  results  are  acceptable.  In  a 
similar  way,  sea  state-dependent  and 
frequency-dependent  surface  Iocs  may  be 
assigned  to  rays  reflecting  off  of  the 

sea's  surface^  (see  ray  "C"  in  Figure 
11).  In  a  shallow  ocean  environment,  the 
extensive  surface  and  bottom  interactions 
require  a  different  approach  altogether 
(see  below). 


More  generally  (other  than  at  the 
bottom  or  at  the  surface),  wherever  the 
sound  speed  gradient  changes  abruptly, 
such  as  at  the  layer  boundaries  in  the  MPC 
Ocean  Model,  there  is  a  chance  that  ray 
tracing  will  break  down.  These  commonlv 
encountered  special  cases  include  the 
phenomena  of  leakage  into  a  shadow  zone, 
intensity  "roll-off"  from  a  cz,  and  a 
surface  duct.  They  are  illustrated  in 
Figure  12,  where  a  plot  of  relative  loss 
has  been  included  to  show  their  effects  on 
transmission  loss.  Reference  to  Equation 
(4)  shows  that  the  deviation  of  these 
effects  from  ray-trace  predictions  are 
more  pronounced  at  lower  frequencies. 


The  MPC  Ocean  Model  adapts  the 

findings  of  research  workers* 17 }  regardinq 
-he  frequency-dependent  and  ssp  gradient- 
dependent  effects  of  a  shadow  zone.  in 
this  adaptation,  the "limiting  rays"  used 
to  define  the  shadow  zone  are  traced,  so 
that  range  dependence  enters  as  it  does 
for  the  rays  in  the  standard  fan. 


The  other  special  cases  are  treated 
by  the  FACT  model  in  an  empirical,  fre¬ 
quency-dependent  manner.  The  asymptotic 
intensity  method  for  roll-off  from  a  cz 
caustic  (from  FACT)  is  modified  and 
limited  to  give  the  basic  effects  in  a 
real-time  framework.  However,  the  actual 
cz  determination  is  an  integral  part  of 
the  MPC  ray-trace  procedure,  and  as  such, 
models  the  range  dependence  not  considered 
by  FACT.  MPC  Ocean  surface  duct  calcula¬ 
tions  also  follow  those  of  FACT.  They  are 
dependent,  however,  on  such  factors  as  the 
sound  speed  gradient  and  the  duct  depth, 
both  of  which  are  allowed  to  vary  across  a 
front  in  the  MPC  Ocean  Model.  Range 
dependence  is  included  through  these 
parameters. 


Pinnacles  also  represent  a  special 
case.  Diffraction  around  a  pinnacle  has 
been  determined  to  be  a  negligible  effect 

for  frequencies  above  25  Hz,*18,19^  so 
that  modified  ray  tracing  can  be  retained. 
As  shadowing  can  be  important,  this  effect 
is  modeled  simply  by  cutting  off  any  rays 
that  intersect  the  pinnacle  (refer  to 
Figure  2 )  . 


MPC  Shallow  Ocean 


Introduction .  The  MPC  Shallow  Ocean 
Model  has  the  task  of  calculating  the  pas¬ 
sive  transmission  loss  when  the  transect 
is  in  water  shallower  than  150  meters.  In 
such  an  environment,  ray-trace  theory  is 
better  replaced  by  the  more  exact  normal - 
mode  theory,  as  has  been  mentioned.  This 
is  effected  by  dropping  the  assumption. 
Equation  (4),  which  led  to  the  eikonal 
equation,  and  returning  to  solve  the  wave 
equation,  Equation  (1),  directly.  To 
tailor  normal-mode  theory  to  fit  a  real¬ 
time  framework,  the  MPC  Ocean  Model  makes 
use  of  the  adiabatic  approximation,  that 
is,  the  range  dependence  is  slow  enough 
that  the  wave  equation  is  "locally  separ¬ 
able,"  and  further,  that  the  modal  solu¬ 
tions  so  obtained  involve  no  inter-coupl¬ 
ing  of  energy.  These  approximations  are 
used  by  almost  every  research  model,  and 

have  been  shown  * 20  *  to  be  quite  good, 
even  for  bottom  slopes  of  up  to  10  de¬ 
grees.  Implementation  in  the  MPC  Ocean 
Model  involves  the  off-line  solution  of  a 
range-independent  problem  at  several  user- 
specified  environmental  sites.  During 
trainer  operation,  the  on-line  module  per¬ 
forms  a  certain  range-average  of  the  pre¬ 
stored  modal  parameters  of  those  environ¬ 
ments  whica  are  crossed  by  the  transect* 


Off-Line  Module.  By  assuming  ideal¬ 
ized  boundary  conditions,  the  velocity 
potential  in  Equation  (1)  is  given  by  a 
certain  sum  of  normal  modes,  or  eigen¬ 
functions  u  ^  (  z  )  ;  each  satisfies  the 

depth-separated  wave  equation: 


where  k  is  the  eigenvalue  or  modal 
n .  (15,16)  m 

wave  number.  This  method  is 

adapted  for  use  by  the  MPC  Shallow  Ocean 
Model,  following  the  Navy  research  model 
(13.14) 

MOATL.  '  For  a  fixed  eigenvalue, 

depth-variations  in  the  sound  speed 
may  cause  the  quantity  [in  brackets  in 
Equation  (5)]  to  vary  from  positive  to 
negative,  in  which  case,  the  character  of 

the  modal  solution  u  will  vary  from 

n 

sinusoidal  to  exponential.  Sample  ssp's 
and  corresponding  normal  modes,  which 
represent  depth  distributions  of  the 
acoustic  pressure,  are  shown  in  Figure  13. 
It  is  evident  there,  tha*  such  features 
as  surface  ducts  and  shadow  zones  are 
intrinsically  modeled  by  the  normal- 
mode  approach.  Cz 1 s  are  also  implicit, 
although  their  occurence  in  the  waveguide¬ 
like  shallow  ocean  is  rare. 


■490  ij.0 


ojch.l*to*y  *no  c**omcntiau 


figure  13.  Normal -Mode  Pressure  Distri¬ 
butions  (sound  speed  uni's  are  m/sec). 

Equation  (5)  is  solved  off-line  for 
each  shallow  ocean  environment,  each  of 
which  is  independent  1  y  defined  by  its 
water  depth,  ssp,  and  sediment  type  and 
•hickness.  A  shallow  ocean  gaming  area  in 
*  lie  East  China  Sea  is  shown  in  Figure  14. 
The  actual  area  (Figure  1 4a)  is  modeled 
(Figure  14b)  using  six  independent  envi¬ 
ronmental  contours,  labeled  with  Homan 
numerals.  For  the  first  four  of  these 
(corresponding  *o  the  transect  shown  in 
figure  }  4b )  .  i  hyj>ot  he'  i  ca  1  model  input  is 


illustrated  in  Figure  15.  The  sediment 
type  names  given  are  standard  in  the 

literature,  as  are  their  properties, ^ ^ ^ 
which  are  supplied  internally  by  the 
off-line  module.  Effects  of  sediment 
variation  have  already  ^en  demonstrated 
in  Figure  6. 


Figure  14.  A  Typical  Shallow  Oct-an  Juming 
Area.  (a)  Actual  area.  (;■)  Simulated 
area.  (Depth  uni's  are  meters.) 
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Figure  15.  Environmental  Parameters  Along 


t  h  e 

T  r  a  n  s  e  c 

t  D  e  f  i 

ned  in  Figure 

1  4b 

For  up 

to  six 

environments , 

Equa- 

tion 

(  5  )  is 

solved 

separately  for 

each 

of  t 

he  four 

ASRAP 

frequencies  (' 

thus, 

frequency-dependent  effects  are  inherent.). 
For  each  of  these  24  cases,  a  maximum  of 
25  normal  modes  are  calculated.  Each 
stored  eigenfield  consists  of  76  depth 
values  of  the  eigenfunction  (normal  mode), 
the  eigenvalue,  and  two  attenuation 
coefficients.  The  total  storage  for  a 
given  gaming  area  is  approximately  forty- 
five  thousand  (16-bit)  words.  The  25-mode 
limit  is  a  fundamental  approximation  jof 
the  MFC  Ocean  Model,  made  to  save  storage 
and  on-line  mode-summation  time.  Although 
the  actual  number  of  modes  supported  by 
the  deepest  environments  for  the  highest 
frequency  can  exceed  400,  truncation  to 
the  lowest-order  25  is  justified  for 

training  purposes  because  all  the  salient 

★ 

features  are  retained. 


The  included  range  dependence  is 
illustrated  by  explaining  the  terms  in 
Equation  (6).  The  terms  HT  and  Hg  are 

the  water  depths  at  the  target  and  sono- 
buoy  locations.  The  use  of  their  product 

*.arI  extensi°n  of  H2,  found  in  the 
flat- bottom  case.  In  the  summation  of 

modes,  Un(T)  and  Ufi(S)  are  similar  range- 

dependence  extensions,  representing  the 
two  different  sets  of  modes  generated  for 
the  target  and  sonobuoy  locations.  As  ir¬ 
is  unlikely  that  the  target  (sonobuoy) 
will  be  located  directly  on  one  of  the 
input  environmental  contours,  the  MPC 
Shallow  Ocean  Model  performs  an  interpo¬ 
lation  between  the  pre-calculated  sets  of 
modes  at  the  two  environments  closest  to 
the  target  (sonobuoy).  The  depth  of  the 


target  is  ;  un(T)(zT)  is  thus,  the 

closest  of  the  76  stored  depth  values  to 
this  depth  (and  similarly  for  the  sonobuoy 

modes)  .  The  term  S  is  t.he  transect  — 

n 

averaged  value  of  the  product  of  the 
eigenvalue  and  the  total  range,  knr  . 

Schematically,  for  the  specific  transect 
shown  in  Firures  14b  and  15,  there  is: 


T  III  II  II  S 


f  r.  ^ 

X  r  V 

>v(T' 

— 

v  2  S 

hi)  k  (in, in 

n 

k  (II> 
n 

t  r4  - > 

k  <«•« 
n 

^  A 

modal  wave  number 


where  k  (i'3)  = 

n 

averaged  between  environments  i  and  j. 

The  term  A  ^  is  a  similarly  transect- 

averaged  value  of  5  r,  wher°  <5  is  t-he 

n  n 

modal  attenuation  factor  and  includes 
water  absorption,  sediment  and  basement 
attenuation,  and  sea  surface  scatter. 
These  depend  on  the  sediment  type,  sea 
state,  etc.,  and  enter  the  theory  via 
perturbation  methods. 


On-Line  Module. 

When  the 

wave  equa- 

tion 

is  solved  directly 

via 

norma 1-mode 

theory,  it  is 

found 

that 

the  transmission 

loss 

is  given, 

under 

the 

adiabc 

l tic  approx- 

imation,  by: 

TL  = 

-l°l°9l0< 

f  2  TT 

1  Vs 

N 

V 

n=l 

[VT><v 

•  u  (S) 

n 

(zs> 

-A 

e  n 

A 

/.  (6) 

1/2 

n 

*  Verified  by  private  calculations,  and 
also  by  Reference  (21). 


The  total  number  of  modes  in  the 
sum  is  N,  which  is  dependent  primarily  on 
frequency  and  water  depth.  The  shallowest 
water  depth  found  Along  the  transect 
is  employed  to  limit  N.  In  this  way, 
sloping  bottoms  are  modeled.  They  are 
also  modeled  implicitly  through  the 
solutions  un,  *n,  6n  at  the  various 

input  environments.  Range  dependence 
in  the  ssp  (or  a  front)  is  similarly 
modeled  implicitly. 

Pinnacles  are  modeled  by  using  their 
top-depths  to  apply  mode  cut-off  (i.e.,  to 
further  truncate  the  value  of  N),  For 
pinnacles  reaching  close  to  the  sea 
surface,  the  lower  frequencies  may  Be 
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completely  cut  off.  For  the  island  in 
Figure  14a  (modeled  by  three  overlapping 
pinnacles  of  zero  top-depth  in  Figure 
14b),  all  frequencies  are  cut  off. 

The  implementation  of  adiabatic  mode 
theory  is  a  quick  and  effective  way  of 
incorporating  range-dependent  shallow 
ocean  environments  into  a  trainer-based 
transmission  loss  model.  Most  of  the  real 
work  [i.e.,  solution  of  Equation  (5)]  may 
be  performed  off-line.  In  the  case  of  the 
MPC  Ocean  Model,  off-line  shallow  ocean 
calculations  consume  about  fen  minutes  per 
qaming  area.  To  calculate  Equation  (6) 
on-line,  takes  an  average  of  40  milli¬ 
seconds  per  transect . 
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ABSTRACT 


Presently  no  set  of  algorithms  has  been  developed  which  assesses  all  variables  mv  'ved  in  deriving  training  dev-ce 
requirements  While  the  Instructional  Systems  Development  (ISDi  process  provides  useful  information  for  deriving 
requirements  for  those  aspects  of  the  trainer  which  are  specific  to  aircraft  configuration,  it  provides  insufficient  information 
for  deriving  best  method"  of  simulation  and  instructional  feature  requirements  The  Systems  Engineering  Process  on  the 
other  hand  is  a  generic  guide  and  does  not  consider  training  variables  If  these  processes  are  used  concurrency,  two 
sepaiate  data  bases  one  training-related  one  engineerino-related  will  evolve  without  any  seeming  correlation  By 
adn  ng  research  design,  the  effect  of  the  training  device  design  upon  training  effectiveness  can  be  assessed  By  collecting 
the  needed  data  based  upon  individual  feature  characteristics  rather  than  implementing  a  single  process  ur  set  of 
algorithms  to  derive  an  feature  requ1  ments  the  training  equipment  contractor  can  develop  equipment  maximizing  training 
anu  cost-effectiveness. 


INTRODUCTION 

Military  instructional  systems  including  simulation  devices  have 
become  very  important  due  to  the  rising  costs  ot  training  personnel  on 
the  actual  equipment  There  is  a  trend  for  rr  Dre  realistic  simulations  ot 
aircraft,  including  associated  airborne  and  weapons  delivery  systems 
This  trend  has  caused  a  corresponding  upward  spiral  in  both  the  initial 
procurement  and  subsequent  life  cycle  costs  associated  with  training 
of  military  personnel  Pohlmann  Isley  and  Caro  '  noted  that  while 
expense  and  fidelity  of  newer  training  equipment  have  increased 
efficient  design  concerning  instructional  features  has  decreased 
having  a  negative  impact  on  the  training  value  of  the  equipment  They 
state.  Because  of  the  cost  of  high  fidelity  devices  the  development  of 
simulator  land  other  training  equipment)  designs  that  permit  more 
efficient  training  is  a  necessary  goal 

The  military  customer  began  to  closely  analyze  training  objectives, 
population  variables  and  other  factors  which  drive  the  effectiveness  of 
equipment  design  The  Air  Force  uses  the  Instructional  Systems 
Development  (ISDi  process  for  that  purpose  Air  Force  Regulation 
50-8  '  defines  ISD  as  a  systematic  but  flexible  process  used  for 
planning,  developing  and  managing  education  and  training  programs 
More  specifically  ISD  sets  guidelines  to  aid  training  managers  in 
identifying  training  requirements  translating  the  requirements  into 
valid  learning  objectives  selecting  proper  training  strategies 
developing  effective  training  delivery  systems  and  providing  quality 
control  Unfortunately  ISO  is  used  before  the  training  equipment  is 
developed  and  thus  does  not  analyze  the  impact  of  instructional 
feature  design  on  the  training  program  The  contractor  usually 
acquires  the  task  of  investigating  the  effects  ot  new  designs  or 
modifications  on  the  learning  situation  and  user  system  interactions 
as  well  as  collecting  any  missing  front  end  data  Traditional  system 
engineering  processes  or  the  sets  of  algorithms  used  for  transforming 
an  operational  need  into  a  descnpfon  of  system  performance 
parameters  and  a  preferred  system  configuration  ''  were  not 
originated  for  use  in  training  situations  and  do  not  provide  means  tor 
collecting  and  analyzing  that  type  of  data  Those  trainer  requirements 
affected  by  training  variables,  then,  which  were  left  for  the  contractor 
to  derive  are  often  based  on  software  hardware  compatibility  and 
other  engineering  considerations  This  results  in  customer- 
contractor  misunderstanding  loss  of  rime  increased  costs  and  often 
a  training  product  which  does  not  fulfill  training  requirements 

The  training  literature  contains  numerous  examp.es  ot  these 
situations  Semple  Cotton  and  Sullivan  >.*?rformed  an  extensive 
review  and  critique  of  instructional  features  for  the  Air  Force  m  which 
they  stated 

An  issue  ot  continuing  concern  is  thai  the  aircrew  instructor  s  Aircrew 
Training  Device  |ATD)  job  seems  to  have  escaped  needed  detailed 
ana'yS'S  The  issue  is  an  Instructional  System  Development  (tSD) 
approach  to  ATD  instructor  training  The  s  Tie  issue  is  of  concern  /  t 
respect  to  console  design  because  consoles  a*e  pieces  of  equipment 


meant  to  support  men  in  the  doing  of  their  jobs  It  is  only  reasonable 
therefore,  to  start  with  a  thorough  knowledge  of  the  tasks  and 
performance  the  instructor  s  job  requires  so  that  consoles  can  be 
designed  to  support  these  requirements  It  probably  is  the  case  that  at 
least  some  ATD  manufacturers  take  the  instructor  s  tasks,  sequences, 
workloading  and  coordination  requirements  into  account  when  they 
design  ATD  instructor  consoles  What  is  needed,  however  are 
improved  guidelines  for  relating  instructor  task  requirements  to 
console  design  characteristics  so  that  what  results  will  be  more 
predictably  workable 

Guidelines  are  needed  tor  incorporating  both  ins’ructor  and  tra  r  ees 
task  variables  into  the  systems  engineering  approach  The  means  .o 
do  this  already  exist,  but  as  Berta'anffv  states  m  General  Systems 
Theory  Modern  science  is  characterized  by  its  eve’-increasing 
specialization  necessitated  by  the  enormous  amount  of  data,  the 
complexity  of  techniques  and  theoretical  structuies  within  every  field 
in  consequence  the  physicist,  the  biologist  the  psychologist  and 
the  social  scientist  are  encapsulated  m  their  private  universes  and 
it  is  difficult  to  get  word  from  one  cocoon  to  the  other  This  paper  wi1 
attempt  to  integrate  existing  approaches  from  education  ttrainmyi. 
psychology  (experimental  anj  learning  theory i  and  systems 
engineering  into  a  singular  working  system  designed  io  derive  training 
equipment  requirements  To  be  more  specific  the  three-  systems 
referred  to  are 

1  The  Instructional  Systems  Development  jtSD)  process  and 
other  front-end  analyses  performed  hy  the  customer 

2  The  scientific  method  including  experimental  designs  and 
hypotheses  revolving  around  learning  theory 

3  The  standard  systems  engineering  process  as  defined  in 
MIL-STD-499A.  Append'*  A 

The  decision  to  combine  these  approaches  was  not  artificially  or 
arbitrarily  imposed  Training  equipment  engineers  have  used  these 
systems  m  an  unstructured  form  before  Clarification  of  the 
interactions  between  the  systems  is  needed  howevei 

As  a  side  note  these  processes  have  two  common  characteristics 
First  ot  an  they  are  systems  designed  to  derive  data  on  which  to  base 
decisions  The  nature  ot  the  information  derived  from  these  systems 
however  is  quite  different  The  mining  device  contractor  must  assess 
each  specific  feature  as  to  the  type  ot  information  needed  to  proauce 
an  efficient  and  effective  design  for  that  specific  comfionent  ot  trie 
training  device  The  contractor  will  need  a  different  set  ot  information 
tor  example  to  denvc  performance  measurement  system 
requirements,  which  is  more  mining  related  than  to  derive  crew 
station  design  tor  a  Weapons  Systems  Trainer  iWSTj  which  is  more 
dependent  upon  the  aircraft  configuration  Montemeno  4  assessed 
t  at  there  are  no  theoretical  or  empirical  bases  for  a  single  set  of 
algorithms  which  can  be  used  to  devetoo  a»  rvograms  ot  instruction 
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Similarly,  we  found  that,  because  the  varying  degree  to  which  the 
contractor  will  employ  each  of  the  different  processes  varies  for  each 
training  equipment  feature,  it  is  impossible  to  provide  a  single  set  of 
algorithms  which  can  be  used  to  derive  all  trainmg  equipment 
requirements 

Secondly,  at  some  point  in  all  three  processes  the  human  must 
make  some  subjective  inputs  In  ISD.  there  is  a  relatively  objective 
method  of  collecting  data  (task  analysis)  but  it  is  still  the  task  of  »he 
training  experts  to  use  that  data  to  determine  objectives,  arrange  a 
syllabus  and  choose  appropriate  media1  Likewise,  the  researche- 
develops  hypotheses,  determines  criteria  and  the  experimental 
design  and  the  engineer  determines  the  method  of  simulating  and 
sets  the  point  at  which  the  cost  of  that  method  outweighs  the  benefit. 
Computer-aided  systems  have  been  developed  to  organize  data  18  . 
but  the  human  must  still  interpret  this  data  The  results  of  training 
equipment  requirements  derivation  process,  therefore,  are  only  as 
good  as  the  human  resources  involved  in  the  loop 

Examples  given  in  the  following  sections  deai  primarily  with  the  Air 
Force  as  the  customer  in  the  procurement  of  an  advanced  WST  This 
system  approach,  however,  ca  he  modified  for  any  customer  and  can 
apply  to  the  entire  family  of  traning  devices,  including  general  framing 
equipment,  maintenance  bench  trainers  integrated  systems 
maintenance  tramps  part  ask  trainers  (PTTs).  mission  trainers 
(MTs).  operational  ...c.;t  framers  (OFTs)  and  v^eapon  system  trainers 

BACKGROUND 

ISD  is  used  at  several  levels  in  the  Air  Force  hierarchy  to  make 
specific  decisions  Ideally  the  process  begins  with  an  analysis  of 
system  requirements  considering  a1!  constraints  For  example  in  the 
Department  of  Defense  tDOLi.  an  analysis  occurs  to  determine  the 
need  for  a  new  long-range  combat  aircraft  At  the  same  time  DOD  or 
AF  Hq  must  analyze  and  define  the  mission  it  is  t*.  perform  At  some 
other  level,  an  analysis  may  occur  to  examine  the  subsystems 
support  systems,  training  equipment  (including  simulators)  personnel 
needs  and  myriad  other  considerations  At  some  point  the  user  of  the 
proposed  simulator  is  brought  into  the  iSD  process  This  group  will 


apply  the  first  step  of  the  ISD  process  by  performing  an  in- depth  task 
analysis  and  target  population  study  The  result  of  this  effort  will  be  a 
definition  of  asi  knowledges  and  skills  to  be  framed,  that  is.  education 
and  training  requirements  The  education  and  training  requirements 
must  then  be  written  as  objectives.  These  objectives  must  state  a 
specific  performance.  *hc  conditions  under  which  the  performance 
takes  place  and  a  standard  for  the  performance'’''  4  Hl 

The  results  of  this  front-end  analysis  are  input  to  the  design  criteria 
data  base  from  which  the  training  device  contractor  designer  applies 
the  systems  enginetmng  process  to  derive  training  equipment 
requirements  (see  Figure  1)  The  data  base  also  includes  all  data 
describing  actual  equipment  (usually  aircraft,  in  our  case) 
configuration,  performance,  stability,  on-board  systems,  etc.,  as  well 
as  data  describing  friendly  and  hostile  environments.  This  kind  of  data 
usually  takes  the  form  of  technical  repods.  tecmcai  orders  (T  Os), 
engineering  drawings,  and  memoranda  of  telephone  conversations 
and  meetings  Ideally  the  results  of  the  ISD  process  should  be 
supplied  in  raw  form  including  training  objectives,  performance, 
conditions  and  standards  of  performance,  and  all  available  task 
analysis  information  Realistically  the  results  are  usually  refined 
into  tra  ning  equipment  requirements  given  to  the  contractor  usually 
in  the  form  of  a  Prime  Item  Development  Specification  (PIDS)  To 
illustrate  where  ISD  is  currently  being  employed  by  the  Air  Force,  the 
process  flow  for  training  equipment  is  iHur’rated  in  Figure  2 


PROBLEM 

The  contractor  can  use  the  data  base  derived  from  the  ISD  process 
to  identify  the  essential  training  media  elements  of  the  framing 
equ.pment  Training  media  features  operationally  defined  a*e 
representations  of  the  actual  equipment  configuration  performance 
characteristics  eic  which  are  needed  for  transfer  of  framing  They 
provide  stimuli  similar  to  those  provided  by  the  actual  equipment 
through  use  of  actual  equipment  components  <crew  station  control 
panels*,  simulation  (motion  basei  or  abstract  representation 
(checklists  for  emergency  procedures  given  on  a  Computer-Aided 


Figure  1.  Relationship  of  ISD  and  Systems 
Engineering  Processes 
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Instruction  (CAI)  system).  While  ISO  can  bo  used  to  answer  “what"  is 
needed  (or  training,  no  methods  are  provided  for  determining  “how’'  to 
represent  what  is  needed.  For  example,  eye  movement  (assessed  in 
the  human  factors  stage  of  ISD)  can  be  translated  into  visual 
simulation  fout-the-window)  requirements  for  brightness,  scene 
contrast  and  system  resolution.  The  method  for  providing  visual 
effects  is  usually  determined  in  the  systems  engineering  process 
which  addresses  variables  such  as  hardware,  software  compatibility, 
costs,  materials,  etc.  A  method  for  assessing  the  effectiveness  of  the 
design  in  meeting  training  requirements  is  not  included,  however,  in 
the  systems  engineering  process  or  the  ISD  process. 

Do!y(,n  suggests  that  the  methods  for  determining  design 
effectiveness  do  exist  in  the  ISD  process,  that  the  real  problem  is  that 
ISD  is  used  only  as  a  front-end  analysis  and  not  throughout  the 
development  of  a  simulation  system.  Recent  studies  within  the  Air 
Force,  however,  have  indicated  that,  in  practice,  the  ISD  process  as  it 
currently  exists  provides  little  systematic  guidance  lor  the  selection  of 
specific  objectives.  Numerous  “considerations"  are  suggested,  but 
little  forma!  structure  is  available  to  make  trade-offs  among  all  of  the 
considerations.  The  PIDS  often  states  simulation  requirements  for  the 
training  media  elements,  but  it  is  questionable  as  to  whether  the 
decision  was  based  upon  assessment  of  design  effectiveness  or  other 
factors.  In  cases  where  “best  method  for  simulation"  has  been 
defined,  the  contractor  will  need  to  implement  research  design  to 
complete  the  data  base  and  perform  formalized  analyses  aimed  at 
maximizing  cost-effectiveness  and  training  effectiveness. 

Regardless  of  the  amount  of  research  employed  by  the  contractor, 
the  ISD  and  front-end  analyses  data  base  is  always  the  starting  point 
for  developing  the  design  of  training  media  olements.  The  front-end 
analysis,  however,  provides  practically  no  information  for  determining 
design  of  those  elements  which  are  more  independent  of  the  aircraft 
configuration,  or  instructional  features.  The  variables  involved  In 
instructional  features  design  are  more  dependent  upon  loarning 
theory  and  perception.  Specifically,  instructional  foaturos  include 
systems  such  as  automatic  briefing  debriefing,  automatic 
demonslration.  automatic  monitoring,  automatic  performance 
measurement,  automatic  record  repiay.  freeze,  mission  file  build,  etc.; 
on  a  broader  scope  instruct'onal  features  can  include  anything  from 
environmental  control  to  instructor  station  configuration. 


RESEARCH  DESIGN  IN  SYSTEMS  ENGINEERING 

Figure  3  illustrates  a  top-level  flow  chad  for  implementing  research 
design  in  training  equipment  design.  As  mentioned  previously,  more 
detailed  approaches  can  only  be  arrived  at  by  assessing  variables 
concerning  the  individual  feature.  Some  considerations  for  developing 
the  detailed  approaches  include: _ 

1.  The  classification  of  the  feature  -  Is  the  feature  a  training 
media  element  or  an  instructional  element  of  the  trainer?  If  it 
is  a  training  media  element,  the  contractor  can  obtain  a  great 
doal  of  vntunblo  knowlodge  for  dosign  from  tho  training 
objectives  and  requirements,  as  well  as  any  task  analyses 
data.  If  the  ISD  data  is  incomplete  or  unavailable,  the 
contractor  may  need  to  perform  any  front-end  analysts 
necessary  to  complete  the  data  base. 

If  the  feature  is  an  instructional  element,  several  methods 
are  available  for  performing  front-end  analyses.  Pohtmann. 
Isiey,  and  Caro  prescribe  a  method  of  creating  design 
guides  for  instructional  features.  The  method  includes 
identifying  the  feature,  defining  the  feature,  its  purposes  and 
intended  use,  function  description,  concurrent  events  and 
diagrams.  It  is  suggested  that  supporting  analysis  (review  of 
previous  studies)  bo  addod  to  this  format  7'  to  account  for 
previous  research  into  the  instructional  feature.  Tho  purpose 
of  this  step  is  for  the  contractor  to  attain  a  full  understanding 
of  the  feature’s  intended  use  in  meeting  training 
requirements  and  to  narrow  down  design  options.  If  there  is 
sufficient  information  in  the  data  base,  if  the  design 
requirements  have  been  defined  explicitly  in  the  PIDS,  or  If 
an  existing  system  satisfies  the  training  requirements 
without  any  problems,  then  the  contractor  may  end  the 
process  with  this  step. 

2.  Revlow  of  oxisflng  systems  -  If  questions  still  exist  aftor 
reviewing  tho  front-ond  analysos,  thon  tho  contractor  may 
need  to  critique  ev  sing  systems  for  any  probloms  in 
meeting  training  and  user  requirements.  Depending  upon 
the  nature  of  the  feature,  the  contractor  may  need  to  perform 
a  neods  assessment  on  tho  training  population  and'or 
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instructor  population,  risk  for  clarification  from  the  customer, 
or  review  nnptioablr?  theories  of  perception  and  learning. 


Figure  3.  Top  Level  Description  of  Training  Equipment 
Design  Method 

3.  Hypothesis  Options  lists  for  design.  There  will 
undoubtedly  be  some  disagreement  as  to  whether  this  step 
should  come  before  or  after  a  review  of  available 
technology.  By  reviewing  technology  first,  the  designer  is 
often  influenced  by  its  limitations.  It  is  suggested,  therefore, 
that  the  designer  become  as  creative  as  possible  in 
developing  solutions  to  meet  the  requirements,  and  then 
search  for  the  technology  which  comes  as  close  as  possible 
to  conforming  to  the  solution,  rather  than  the  design 
conforming  to  the  technology. 

4.  Experimental  design  development  The  contractor  has  the 
most  flexibility  at  this  stage  in  setting  the  criteria  for  success. 
The  criteria  must  be  relevant  to  the  goal,  and  the  designers 
should  ask  themselves:  How  can  a  decision  system  be 
tested  as  to  its  goal  fulfillment  and  efficiency?  Does  the 
system  fulfil)  i*s  purpose  at  a  responsible  degree?  Does  it 
satisfy  the  minima!  criteria  set  for  it?  Shall  it  be  accepted, 
revised  or  rejected?  There  exists  marked  differences 
between  the  testing  of  cognitive  hypothesis  and  theories  and 
the  testing  of  instrumental  hypothesis  and  systems,  as 


illustrated  in  Figure  4.  The  designers  must  bo  av,«irc>  that 
they  are  setting  tho  criteria  for  one  system  tailored  for  one 
set  of  training  requirements  and  not  trying  to  create  a  set  of 
■  truths"  to  meet  a!)  trainer  requirements 


Cognitive  Hypotheses 
(Dominates  in  Pure  Science) 

Instrumental  Hypotheses 
(Dominates  In  Applied 
Science) 

Structure  of  the  general 
form:  “Ail  A  are  B“  and 
variations 

Structure  of  the  general 
form:  “To  attain  A,  do  B" 
and  variations 

Non-teleologic 

Goal  oriented 

Efficiency  irresponsive 

Highly  efficiency  responsive 

Rigorous  criteria  of 
acceptance  based  on  truth 
assumption 

Relaxed  criteria  of 
acceptance  based  on 
assumption  of  better  goal 
attainment 

High  degree  of  generality 

Limited  degree  of  generality 

Not  behaviorally  limited 

Predominantly  oriented 
toward  decision  behavior 

Oriented  toward  cause  and 
effect  relations 

Oriented  also  toward  reason 
and  action  relations  j 

Figure  4.  Some  Characteristics  Distinguishing  Cognitive 
from  Instrumental  Hypotheses 

The  method  of  testing  must  also  be  relevant  for  answering 
unresolved  requirements.  For  so  long  the  training 
community  has  felt  that  task- analysis  was  the  tool,  it  is 
suggested  for  use  in  determining  function  allocation  in  the 
systems  engineering  process  and  for  determining  training 
requirements  during  the  ISD  process.  That  method  usually 
works  only  tor  the  training  media  elements  of  the  trainer 
where  the  aircraft  configuration  is  well  defined.  For 
instructional  feature  design  assessment,  task  analysis  may 
become  too  cumbersome  or,  oven  worse, 
counterproductive.  Folley'*’  stated  that  the  variety  ol  tasks, 
and  the  complexity  of  human  behavior,  would  not  permit  the 
reduction  of  task  analysis  to  a  simple  routine  method  It  may 
be  bettor,  for  instructional  foatuio  design,  to  assess 
instructor  acceptance,  operation  time,  training  time  and 
overal!  training  effectiveness  and  efficiency. 

5.  Cost-effectiveness  After  the  results  of  the  research  he  .re 
been  identified,  the  systems  engineering  process  can  be 
used  to  determine  cost-effectiveness,  technology 
availability,  etc. 

There  are  several  words  of  warning  about  using  this  system  of 
which  the  designer  should  be  aware.  This  method  falls  into  the  realm 
of  applied  science.  Mattessichf,?’  differentiates  between  pure  and 
applied  science: 

“Both  pure  and  applied  sciences  fulfill  a  cognitive  as  well  as  an 
instrumental  task,  but  in  each  branch  these  two  ingredients  are  mixed 
in  different  proportions.  Roughly  speaking,  one  might  say  that  the 
cognitive  element  domlnatos  puro  science:  All  ol  its  statements  havo 
a  sufficiently  high  dogroe  of  reliability  to  be  assumed  true,  but  their 
specific  usefulness  may  not  be  established.  Whereas  the  instrumental 
element  dominates  applied  scienco:  Aii  of  its  statements  are 
assumed  to  be  useful  but  their  degree  of  reliability  is  not  necessarily 
high  enough  to  regard  them  as  ttue  in  the  conventional  senso  Thus, 
the  applied  sciences  use  the  same  methods  as  tho  pure  ones 
(observation  and  measu-ement.  Induction  and  deduction. 
Interpretation  and  testing,  etc,)  but  with  a  fairly  spoclflc  purpose  in 
mind,  and  under  consideration  of  an  oconomic  or  cost-benefit 
criterion.  Tho  distinction  botween  pure  and  applied  sclonce  is  not 
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tK(!  between  knowing  and  doing.’  but  rather  that  between 
ooi'eving  for  the  sake  of  knowing  and  ‘behoving  for  the  sake  of 
lining  " 

TMrO  t.-oonomic  element,  assessed  through  the  systems  engineerin') 
process  w*!l.  cl  course,  influence  !ho  selection  oh  the  final  feature 
design.  It  is  important,  however,  that  designs  winch  proved  to  be 
effective  but  not  used  are  documented,  as  tochncogies  may  later 
develop  which  will  make  those  designs  feasible. 

Certain  pure’'  sciences  are  taken  into  consideration  in  this 
"applied"  science  approach.  The  contractor  may  need  to  take 
responsibility  tor  collecting  the  pure  research  data,  too.  rather  than 
relying  upon  “known"  theory,  particularly  when  dealing  with 
instructional  features.  As  McKeachie"3’  states.  "The  past  two  years 
have  boon  bad  ones  for  those  of  us  who  have  attempted  to  applv 
traditional  principles  of  'earning  to  instruction.  Thorndike's  principles 
of  learning  seem  to  be  crumbling."  The  instructional  feature  designer 
may  no  longer  be  able  to  assume,  for  example,  that  immediate 
feedback  (automatic  cueing)  will  result  in  more  effective  learning  in  all 
training  situations. 

Likewise,  the  contractor  may  need  to  take  responsibly  for 
performing  the  front-end  analysis,  rather  than  expecting  to  receive  the 
information  from  the  customer.  Pohlmann.  et  al  state  that: 

"II  is  inconceivable  that  one  would  expect  u  designer  to  design  a  flight 
training  simulator  without  giving  him  (or  her)  a  great  deal  of 
information  about  the  intended  instructional  activities.  Yet,  it  :s 
apparent  from  inspection  of  numerous  existing  simulators,  from 
review  oi  simulator  design  procedures,  and  from  review  of  the  relevant 
literature  that  designers  typically  are  given  very  little  information  about 
the  instructional  activities  intended  to  be  used  with  the  device  they  are 
to  design  and  the  functional  purposes  of  these  activities." 

Much  of  this  problem  can  be  accounted  for  due  to  the  traditional  role 
allocation:  The  customer  is  responsible  for  all  of  the  front-end 
analysis,  the  contractor  for  the  systems  engineering.  This  situation  is 
an  idealism,  not  a  reality.  In  order  for  the  proposed  model  to  work, 
constant  feedback  must  occur  between  the  customer  and  contractor, 
and  both  must  take  responsibility  for  data  collection.  If  systems  were 
perfect  and  al!  >nputs  were  known  and  controllable,  all  processes 
known  and  intended,  and  no  selection  and  distortion  took  nlace.  then 
all  outputs  would  bo  known  and  anticipated,  and  the  system  could  be 
allowed  to  run  with  no  mechanism  to  monitor  its  behavior  (feedback). 
However,  no  system  is  perfect.  This  is  why  it  is  imperative  that  the 
contractor  employ  personnel  who  are  knowledgeable  about  current 
trends  in  education  and  training,  and  the  government  assign 
hardware  software-oriented  personnel  to  work  with  the  contractor. 
Feedback  cannot  exist  if  both  parties  speak  different  .^nguages. 

CONTINUING  DESIGN  VALIDATION  EFFORTS 

The  top-leve!  flow  chart  (Figure  3)  represents  a  model  for  use  during 
the  requirements  definition  stage  for  individual  features.  The 
contractor  should  also  assess  the  overall  effectiveness  of  the  training 
equipment  once  the  optimum  design  *or  the  components  has  been 
derived.  Adams"’  identified  three  ap '•■'cations  for  flight  simulators: 
research,  evaluation  of  performance,  and  training.  “Research"  implies 
the  more  "pure"  science  which  was  mentioned  above,  including 
transferability,  perception,  effects  of  cueing,  etc.  Evaluation  of 
performance  and  training  lor  the  contractor  must  also  be  assessed 
through  “applied”  research.  This  may  appear  as  an  exercise  in  futility 
since  the  product  has  already  boon  designed  and  is  ready  for  use. 
However,  the  purpose  is  to  collect  more  information  to  add  to  tho 
training  equipment  data  base  for  reference  for  future  trainers. 

Those  conditions  appear  to  place  an  extreme  amount  of 
responsibility  on  tho  contractor.  However,  the  government  must  also 
play  its  part  in  providing  more  specific  direction  to  the  contractor 
regarding  the  product  which  is  to  bo  developed  and  in  giving  the 
contractor  adequate  time  for  developing  the  training  equipment, ,3>  If 
this  system  is  implemented  to  its  fullest  extent,  then  equipment 
requirements  derivation  will  bo  less  time-consuming  in  the  long  run  as 


more  information  is  being  collected  iri  the  data  base  Perhaps,  then. 

tho  day  will  come  when  the  contractor  noed  only  sit  a!  tho  CRT,  input 

the  training  requirements,  and  automatically  receive  tho  most  cost¬ 
and  training-effective  equipment  requirements  and  design. 
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Current  instructional  systems  development  (LSD)  techniques  are  not  well  equipped  to 
uentny  and  o.feet-voly  provide  for  team  training  requirements.  While  much  research  aimed 
at  extending  the  1SD  mode!  to  team  training  development  is  in  progress,  a  systematic 
approach  to  meet  near-term  team  training  development  needs  is  necessary.  This  paper 
provides  an  overview  of  an  approach  we  have  formulated  and  used  successfully  to  this  end. 
Inc  approach  is  based  on  a  functional  analysis  of  the  system  goals/objectives  which  the 
team  performance  in  question  supports.  The  results  are  then  represented  systematically  in  a 
process  model  designed  to  capture  t.;e  dynamic  relationships  between  system  conditions,  and 
categoriCto  team  performances.  The  model  is  used  to  develop  scenario-based  exercise 
guides,  similar  to  instructor  guides  for  classroom  use,  but  designed  to  provide  instructor 
guidance  regarding  training  device  employment  in  the  administration  of  effective  team 
training. . 


INTRODUCTION 

Methods  for  the  systematic  determination  of  indi¬ 
vidual  skills  are  currently  available.  Training  objec¬ 
tives  (i.c.,  objective  behavioral  statements,  conditions 
of  performance,  and  performance  standards/criteria) 
derived  using  these  methods  tend  to  be  well  defined. 
Subsequent  training  system  development  and  implemen¬ 
tation  on  this  base  is  relatively  straightforward.  Col¬ 
lectively,  these  methods  constitute  the  instructional 
systems  development  (ISD)  model  in  all  its  manifesta¬ 
tions  (?.g.,  NAVCDTRA  106A/TRADOC  350,  MIL-T- 
290  53B(TD),  MIL-STD- 137913,  NAVEDTRA  110, 
NAVSEA  OD  05519).  Application  of  the  ISD  model  to 
team  skill  determination,  however,  has  not  been  equally 
eflective.  The  literature  attests  to  this  observation 
repeatedly  ( 1 3C 2)( 33C**3.  The  principal  difficulty,  en¬ 
countered  early  in  the  ISD  process,  lies  in  identifying 
team  skills  (or  training  rccuiirements,  or  training  objec¬ 
tives)  with  the  same  precision  and  objectivity  as  is 
possible  with  individual  skills.  As  a  result,  the  systema¬ 
tic  measurement  of  team  performance  is  elusive. 
Without  systematic  performance  measurement,  the  pro¬ 
vision  of  adequate  feedback  to  the  team  is  hampered. 
This  inability  to  close  the  so-called  training  cycle  by 
means  of  adequate  performance  feedback  leads  to  inef¬ 
ficient  and  in  many  cases  ineffective  team  training. 

So  saying,  however,  it  is  noted  that  the  ISD  model 
is  one  of  the  more  useful  developments  of  instructional 
Technology.  The  intent  underlying  the  model  is  to 
ensure  the  development  of  an  effective  training  system. 
That  is,  one  which  provides  trainees  through  its  opera¬ 
tion,  the  knowledges  and  skills  necessary  to  meet  the 
initiating  training  requirement.  This  is  accomplished  by 
the  systematic  application  of  defined  planning,  analysis, 
development,  implementation,  and  evaluation  proce¬ 
dures.  The  ISD  model,  however,  is  only  a  guide.  Its 
virtue  lies  in  the  fact  that  it  forces  the  systematic 
consideration  and  orderly  development  of  each  training 
system  component.  The  principal  drawback  lies  not 
within  the  model,  but  in  the  literal  interpretation  or 
uninformed  application  of  the  procedures  described. 
This  is  particularly  true  with  respect  to  team  training 
development. 


To  counter  this  drawback,  many  efforts  to 
formally  extend  the  ISD  model  to  team  training  have 
been  reported  (e.g.,  4).  For  th<“  most  part,  these  have 
concentrated  on  the  fundamental  problems  of  team 
performance  definition  and  measurement.  Progress  in 
these  areas  is  apparent.  For  example,  recent  ^concep¬ 
tualizations  of  team  performance  explicitly  re¬ 

cognize  the  dependencies  among  individual  and  team 
performance,  team  tasks  and  the  situation  in  which  that 
performance  occurs.  However,  while  these  efforts  ate 
prcmis.ug,  researchers  in  the  area  acknowledge  that 
further  conceptual  work  is  required  before  specific 
team  skills  and  their  interrelationships  can  be  isolated 
with  the  precision  necessary  for  the  derivation  of 
explicit  training  requirements.  Progress  in  the  area  of 
team  performance  measurement  has  also  been  reported. 
For  example,  Connelly  and  his  associates  have 
developed  an  approach  which  permits  the  evaluation  of 
both  team  and/or  individual  team  member  performance 
in  an  ongoing  mission  context  (6j.  This  includes  specific 
tasks  and  task  types.  Again  while  the  methodology 
appears  promising,  they  point  out  that  additional  work 
in  the  area  of  generic  task  specification  is  required. 
Therefore,  the  systematic  development  of  team  train¬ 
ing  in  the  classic  sense  continues  to  be  an  elusive  goal. 

In  the  meantime,  the  quantity  of  resources  which 
have  been  and  are  being  committed  to  team  training 
system  and  device  development  is  large.  The  need  for 
an  ISD-based  approach  was  never  clearer.  Our  efforts 
in  the  area  of  submarine  team  operational  training  have 
made  this  point  particularly  clear  to  us.  The  lack  of  an 
ISD-based  approach  has  led  us  to  the  formulation  of  an 
ISD-iike  process  for  team  training  material  develop¬ 
ment,  an  overview  of  which  is  presented  in  this  paper. 
The  approach  we  took  was  pragmatic.  Our  intent  was 
that  it  be  based  on  ISD  principles,  that  it  accommodate 
findings  of  research  yet  to  be  completed,  and  at  the 
same  time  that  it  provide  a  useful  tool  for  the 
development  of  team  training  materials  to  meet  the 
near-term  requirement,  It  is  based  on  the  following 
four  considerations. 
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BACKGROUND  CONSIDERATIONS 

I  irst,  ISD  analytical  processes  grew  out  of  u 
behavioral  psychological  tradition,  focused  on  individual 
performance,  ant!  originally  addressed  operator  and 
maintenance  activities  in  well-defined'  (established) 
situations.  While  the  distinction  may  not  be  exclusive, 
team  performance  in  military  systems  is  more 
accurately  described  as  operational  in  nature,  focusing 
on  the  functional  objective  the  system  (the  equipment 
and  the  team  personnel)  was  designed  to  meet.  As 
such,  team  performance  involves  problem  solving, 
resource  allocation,  and  decision  making  processes  on  a 
different  scale  and  in  less  structured,  dynamic 
situations  than  for  isolated  individual  u.:tivitv.  The 
atomization  of  performance  when  employing  classical 
analysis  methods  tends  to  obscure  the  relationship  of 
each  component  task  or  element  to  the  overall  system 
objective.  This  observation  suggests  an  approach  based 
on  a  functional  analysis  of  the  system  and  system 
objective  of  which  the  team  to  be  trained  is  a  part. 

Second,  the  requisite  individual  skills  collectively 
required  to  support  successful  team  performance,  (in 
terms  of  the  system  functional  objective)  may  be,  and 
usually  are,  distributed  differently  among  individual 
team  members  in  different  teams.  Further,  this 
distribution  changes  with  experience.  From  the 
training  development  point  of  view,  this  makes  it 
extremely  difficult  to  specify  generic  team  training 
requirements  which  are  valid  for  ali  teams.  That  is, 
what  is  a  correct  procedural  training  requirement  for 
one  team  may  not  be  for  a  second  team,  though  the 
second  team  may  evidence  the  same  summativc 
performance  deficiencies.  The  ISD  model  is  not 
currently  constituted  to  provide  for  this  type  of 
variability.  Consequently,  we  formulated  our  approach 
to  provide  a  training  requirements  structure  within 
which  teum-to-team  performance  differences  could  be 
addressed  flexibly.  In  so  doing,  however,  greater 
emphasis  is  placed  on  the  instructor's  role.  This 
consideration  was  also  addressed  in  the  formulation  of 
our  approach. 

Tmrd,  the  decision-making  tasks  at  various  levels 
within  a  team  structure  arc  action  oriented,  finite  in 
number,  and  emphasize  resource  management.  The  key 
variables  are  associated  with  the  information  available 
(static  and  situational)  on  which  to  base  a  decision: 
amount,  quality,  rate,  timeliness,  and  the  set  of 
alternative  decisions  they  define.  These  are  task 
conditions,  not  tasks  per  se.  This  suggests  that  an  ISD- 
like  development  of  team  task  listings  should  be 
relatively  circumscribed  with  greater  emphasis  placed 
on  the  delineating  conditions  (operational  situations) 
affecting  the  information  flow  and  individual 
responsibilities/authority  (formal  and  informal)  within 
the  team  structure.  (This  is  indirectly  consistent  with 
the  observation  that  the  stimulus-response  behavioral 
basis  of  classical  ISD  analysis  rnay  not  be  suited  to 
team  performance  analysis.  Instead,  a  cognitive  basis 
may  be  more  appropriate.  This  mirrois  the  growing 
recognition  of  the  shift  in  the  field  of  learning 
psychology  away  from  behaviorism  toward  a  cognitive 
basis  (T).) 

Fourth,  the  administration  of  team  training 
because  of  the  dynamic,  operational  aspects  of  team 
performance,  generally  depends  on  an  operationally 
realistic  training  environment.  This  is  typically 
provided  by  conducting  training  exercises  using  a 
simulator  or  training  device.  These  have  become 
complex  and  costly,  driven  by  the  belief  that  team 
training  transfer  to  the  operational  context  is 


particularly  sensitive  to  the  fidelity  of  the  training 
situation  to  the  operational  context.  Certainly,  the  ISD 
process  should  play  a  significant  role  in  balancing 
optimum  trainer  capability  against  cost.  The  process 
should  not  only  define  trainer  functional  requirements, 
but  also  provide  a  basis  for  prioritising  them,  first  in 
terms  of  the  initiating  training  requirement,  and  second 
in  terms  of  trainer/training  effectiveness.  This 
interaction  is  a  rather  sophisticated  aspect  of  the  ISD 
media  selection  process  which  is  not  fully  documented. 
However,  the  instructor  who  must  use  the  training 
device  is,  from  our  perspective,  mere  important.  As 
noted  above,  the  instructor's  role  in  team  training  is  of 
considerably  more  significance  than  for  individual 
training.  The  team  trainees  will  be  dynamically 
exercising  previously  learned  knowledge  and  skills  in 
support  of  the  system  functional  objective.  Student 
training  materials  are  not  a  part  of  this  process. 
Instead,  the  instructor  monitors,  evaluates,  and 
critiques  the  team’s  performance  and  in  general 
mediates  the  training  process.  This  includes 
employment  of  the  training  device  to  best  effect.  For 
this,  he  requires  guidance.jof  a  different  nature  than 
that  found  in  instructor  guides  supporting  classroom 
instruction.  Further,  it  is  not  sufficient  to  train 
instructors  to  operate  the  training  device  only.  In  the 
same  sense  that,  for  example,  a  sonar  operator  must  be 
trained  in  the  operational  employment  of  his  equipment 
(i.e.,  in  addition  to  Operator  &  Maintenance  training)  to 
realize  its  functional  capability  in  the  tactical  context, 
so  must  the  instructor  be  trained  m  the  employment  of 
his  training  device  to  deliver  effective  operational 
training.  Therefore,  the  material  provided  for  his 
support  should  guide  his  employment  of  the  training 
device  for  effective  team  training  during  each  training 
session  or  exercise.  This  aspect  of  team  training 
system  development  is  crucial.  Further  it  can  become 
quite  involved,  in  effect  requiring  a  mini-lSD 
development  depending  on  the  complexity  of  the 
team/system  function,  training  device,  and  the  number 
of  instructors  involved  in  its  operation  during  a  training 
session.  Uninformed  application  of  the  ISD  process 
often  fails  to  adequately  treat  this  aspect  of 
operational  training  system  development.  Conse¬ 
quently,  we  formulated  our  approach  to  focus  on 
supporting  the  instructor  in  his  administration  of 
effective  team  training. 

PROCESS  DESCRIPTION 

Our  approach  to  team  training  development 
recognized  these  factors.  In  particular,  it  explicitly 
recognizes  the  role  of  the  instructor  as  the  catalyst  (or 
effective  team  training  and  focuses  on  providing 'him 
with  the  necessary  support.  Accordingly,  the  principal 
product  is  an  exercise  guide,  similar  to  an  instructor 
guide,  but  differing  in  context  and  format.  The 
approach  involves  seven  steps: 

o  Data  base  establishment 
o  Exercise  goal/objective  development 
o  Process  mode!  development  and  operation 
o  Exercise  selection 
o  Exercise  development 
o  Quality  control 
o  Validation 

Activities  involved  in  each  step  are  described  below  in 
terms  of  a  specific  application  to  submarine  piloting 
team  training. 


B; 


iv 


308 


0%'ita  Base  E s_i_a blishmet i f 

Information  on  which  the  team  training  development 
process  depended  was  gathered  and  consolidated  in  a 
data  base.  This  information  was  organized  ir,  terms  of 
genera]  and  platform-specific  piloting  procedures  and 
guidelines,  platform-specific  navigation  equipment  as  it 
pertains  to  piloting,  the  associated  trainer  capabilities 
and  operation,  training  material  requirements  and  spec¬ 
ifications,  and  platform-specific  piloting  pipeline  train¬ 
ing.  Analysis  of  these  data  established  initial  starting 
points  for  material  development  in  three  critical  areas: 

(1)  exercise  goals  and  objectives,  (2)  material  format 
and  Organization,  and  O)  instructional  guidance.  Subse¬ 
quent  development  drew  on  this  data  base  is  required. 

Exercise  C.oal/Objective  Development 

Exorcise  goal/object ive  development  began  with  a  top 
down,  functional  analysts  of  the  piloting  tearn/systern 
objective.  That  is,  navigation  has  been  defined  as  the 
process  of  safety  (standard)  directing  (activity)  a  ship 
from  one  point  to  another  (platform  operation).  The 
activity  of  directing  involves  three  categories  of 
genera!  team  activities:  planning  own-ship  track,  re¬ 
petitively  fixing  own-ship  position,  and  evaluating  fixed 
position  against  intended  track  as  a  basis  for  recom¬ 
mending  own-ship  maneuvers.  While  the  general  stan¬ 
dard,  team  activity,  and  platform  operation  remain  the 
same,  the  marr.er  in  which  each  general  activity  is 
performed  varies  as  a  function  of  details  regarding  the 
platform  operation  and  conditions  under  which  it  is  to 
be  performed.  Far  example,  specifying  that  the  plat¬ 
form  operation  is  to  be  carried  out  in  restricted  waters 
(condition)  identifies  those  specific  activities  which 
must  be  performed  in  each  category  associated  with 
piloting,  as  well  as  the  detailed  standards  to  which  each 
activity  must  be  performed  in  order  to  safely  complete 
the  platform  operation.  As  the  first  step  in  the 
exercise  goal /objective  development  process,  this  type 
of  general -to-spccific  analysis  was  performed  on  the 
data  base,  beginning  at  the  piloting  (rather  than  the 
more  genera!  navigation)  level.  Piloting  team  activi¬ 
ties/standards  in  each  of  the  three  general  categories, 
platform  operations,  and  conditions  were  identified  and 
organized.  The  process  resulted  in  the  following  organ¬ 
ization  of  information. 

a.  Platform  Operation.  The  principal  platform  oper¬ 
ation  to  be  accomplished  in  a  piloting  situation  is 
transiting  from  point  A  to  point  B.  At  point  B  the 
operation  is  completed  at  a  specific  geographic 
location  such  as  an  anchorage  or  a  destination. 

b.  General  Team  Activity.  The  performance  of  a 
general  team  activity  is  required  in  accomplishing 
a  platform  operation.  As  for  navigation  in 
general,  there  are  three  of  these  executed  repeti¬ 
tively  during  piloting  operations:  planning  own- 
ship  track,  fixing  own-ship  position,  and  deriving 
recommendations  for  own-ship  maneuvers  from  an 
evaluation  of  fixed  position  against  intended 
track. 

c.  Condition.  A  condition  is  a  factor  affecting  the 
manner  in  which  the  general  team  activities  re¬ 
quired  to  successfully  accomplish  a  platform 
operation  are  carried  out.  Conditions  were  organ-  “  ' 
ized  into  three  categories: 

l.  Conditions  affecting  the  accuracy  with  which 
own-ship’s  position  is  fixed  such  as  bearing 
error,  radar  degradation,  etc. 


ii.  Conditions  causing  own-ship  to  deviate 

from  intended  track  in  the  course  of  an 
operation,  such  as  set  and  drift,  contact 
encounters,  propulsion  malfunction,  etc. 

‘ii*  Conditions  requiring  accelerated  perform¬ 

ance  rates  and/or  greater  precision  such  as 
warnings  of  approaching  danger,  proximity 
to  shoal  waters,  submerged  operations,  etc. 

d.  Exercise  Goal.  An  exercise  goal  described  the 
general  training  content.  ft  is  a  descriptive 
statement  which  specifies  the  team  to  be  trained, 
the  platform  operation  which  training  supports, 
and  the  conditions  under  which  the  operation  is  to 
be  conducted. 

c.  Specific  Team  Activity.  A  specific  team  activity 
is  one  that  is  required  in  accomplishing  the  exer¬ 
cise  goal  under  the  specified  conditions.  It  falls 
within  one  of  the  three  general  team  activity 
categories. 

f.  Exorcise  Objective.  Satisfactory  performances  of 
the  required  specific  team  activities  in  an  exor¬ 
cise  are  the  exercise  objectives.  Satisfactory 
performance  is  determined  by  appropriate  stan¬ 
dards. 

g.  Standard.  A  statement  of  the  precision,  rirrteli- 
ness,  and/or  sequence  with  which  a  spool;-  team 
activity  must  be  performed  under  the  specified 
conditions  in  order  for  the  platform  to  safely  and 
successfully  complete  the  operation. 


Process  Model  Development  and  Operation 

The  next  critical  step  in  the  process  involved  develop¬ 
ment  of  a  general  model  of  the  piloting  process.  The 
model,  constructed  during  the  analysis  of  ooerational 
information  in  the  data  base,  describes  the  essential 
interrelationships  among  (1)  platform  piloting  opera¬ 
tions,  (2)  piloting  team  activities  in  each  category,  and 
(3)  conditions.  If  a  platform  piloting  operation  (transit 
from  point  A  to  anchorage  at  point  B)  and  a  set  of 
conditions  (low  visibility)  are  selected  and  provided  to 
the  model  as  inputs,  the  model  will  select  the  appro¬ 
priate  team  activities  which  are  required,  satisfactory 
performance  of  the  selected  activities  thus  becomes 
the  behavioral  objective  of  training.  The  standard  for 
satisfactory  performance  is  derived  from  successful 
accomplishment  of  the  operation.  In  terms  of  an 
exercise,  the  selected  platform  operation  and  condi¬ 
tions  determine  the  exercise  goal  and  the  exercise 
scenario.  The  model,  operating  on  those  same  data, 
will  identify  the  associated  team  activities  required  in 
order  for  the  platform  to  attain  the  exercise  goal. 
These  activities,  together  with  standards  for  their 
satisfactory  performance,  become  the  exercise  objec¬ 
tives.  The  process  thus  resulted  in  the  following 
organization  of  information. 

Exercise  Selection 

Unlike  the  ISD  model  which  addresses  the  selection  of 
tasks  for  training  at  this  poiut,  our  approach  next 
focused  on  the  seiection  of  exercise  goals,  as  defined 
above,  for  exercise  guide  development.  This  approach 
was  taken  because  the  team  activities  identified  arc 
recurrent  and  are  common  to  most  piloting  situations. 
Therefore,  the  intent  was  to  ensure  that  the  resulting 
exercise  set  (22  as  it  turned  out)  'as  sufficiently  broad 
to  ensure  the  trainee  team  the  opportunity  to  exercise 
specific  team  activities  under  all  important  piloting 
circumstances  identified  by  the  piloting  process  model. 
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Exercise  Guide  Dove* lop » n on t 

’Ihei  fifth  step  in  the  process  was  to  choose  an  exercise 
Koai  from  those  selected  in  Step  U.  An  operational 
scenario  was  then  constructed  based  on  the  goal  and  the 
associated  conditions  such  that  during  the  course  of  the 
exercise  those  team  activities  related  to  the  objectives 
would  normally  be  expected  to  occur.  The  scenario  was 
then  modified  with  respect  to  training  device  capabili- 
tics/lirnitations  and  annotated  with  required  device 
control  information.  Training  and  performance  evalua¬ 
tion  information  was  then  added  with  respect  to 
scenario  events.  This  information  consisted  of  the 
required  team  procedures/per formances,  interactions, 
and  outputs  related  to  the  selected  exercise  objectives. 
The  draft  exercise  guide  was  then  entered  into  word 
processing,  reviewed  for  compliance  with  approved  for¬ 
mat  specifications,  and  retyped  in  preparation  for 
review. 


regarding  tailoring  of  the  approach  to  particular  appli¬ 
cations  (e.g.,  other  teams)  is  required.  For  example, 
the  procedure  for  developing  process  models,  a  key 
feature  of  this  approach,  requires  full  explanation, 
what  is  presented  here  represents  only  an  overview. 
Further,  our  approach  does  not  intend  to  answer  out¬ 
standing  questions  regarding  the  theoretical  aspects  of 
team  performance  as  related  to  training.  It  does, 
however,  provide  a  workable  framework  within  which 
these  answers  can  be  incorporated  into  team  training 
development  as  they  become  available.  In  the  interim 
it  provides  a  basis  lor  developing  effective  support  for 
the  team  training  instructor.  The  approach  has  proven 
useful  in  eight  different  applications  to  date  involving 
the  development  of  instructor  materials  supporting  both 
shorebased  and  shipboard  team  operational  training. 
The  products  —  exercise  guides  —  have  been  well 
received  as  indicated  by  feedback  from  the  end  user. 


Ouaiitv  Control 
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Ihe  final  step  m  the  development  process  involved 
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‘  it  ion.  Inc  purpose  was  to  ensure  that  events 
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setting  up  the  training  device  as  required  by  the  exer¬ 
cise  guide  and  stepping  through  the  exercise  scenario  in 
event  sequence.  At  each  step  the  training  device  was 
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incorporated,  anti  subsequently  delivered  for  Navy  use. 


DISCUSSION 

The  process  described  herein  is  presented  as  an 
approach  not  a  model.  It  represents  an  extension  of  1SD 
principles.  Further  documentation  of  guidance 
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ABSTRACT 


Collective  front-end  analysis  is  the  process  bv  which  the  critical  missions  and 
collective  tasks  of  a  battalion  are  specified.  Collective  tasks  are  units  of  work  rc~ 
qu.rfnt;  two  or  more  people  for  their  completion.  A  mission-based  approach  is  employed 
in  C.H-.A.  Iho  CFKA  process  begins  with  specification  of  a  battalion  mission,  proceeds 
to  spec  I ! icat ion  oi  battalion  element  missions,  and  ends  with  delineation  and  description 
° 1  colll‘ctivo  tasks-  Th*  mission-based  approach  to  CFF.A  is  advantageous  because  it  helps 
ensure  a  thorough  listing  of  a  unit’s  collective  tasks  and  provides  a  means  of  relating 
t.i.ik  criticality  directly  to  a  unit  mission.  In  order  to  reduce  the  workload  on  the  user 
and  help  organize  the  massive  amounts  of  data  involved,  user  job  aids  were  prepared. 


INTRODUCTION 


Most  scenarios  for  full  scale  confrontation 
between  the  United  States  and  any  of  its  major 
adversaries  describe  a  "come  as  you  are  war"  in 
which  the  U.S.  will  have  to  fight  immediately 
without  tiie  luxury  of  a  lengthy  mobilization 
period  as  enjoyed  in  previous  wars.  This  rapid 
response  environment  in  which  U.S.  forces  must 
exist  requires  that  fighting  units  establish  and 
maintain  high  readiness  levels.  An  important 
factor  influencing  a  unit’s  readiness  is  the 
training  it  receives.  Development  of  effective 
unit  training  is  predicated  upon  the  ability  to 
identify  performances  that  are  critical  to  ac¬ 
complishment  of  the  unit's  mission.  Among  the 
performances  that  support  accomplishment  of  a 
unit's  mission  are  many  tasks  that  require  teams 
or  collectives  of  personnel  for  their  performance. 
Traditionally,  front-end  analysis  efforts  have 
failed  to  address  collective  tasks  or  have 
addressed  them  inadequately.  As  a  result,  train¬ 
ing  of  collective  or  team  performance  in  units 
has  suffered. 

Recognizing  the  importance  of  team  perform¬ 
ance  to  unit  readiness,  the  Army  lias  recently 
been  concerned  with  developing  a  means  of  identi¬ 
fying  collective  tasks.  As  defined  by  TRADOC 
PAM  310-8,  collective  front-end  analysis 
(CFEA)  is  the  process  by  which  the  critical  mis¬ 
sions  and  collective  tasks  of  a  battalion  are 
specified.  This  paper  provides  a  brief  descrip¬ 
tion  of  an  approach  to  CFEA,  discusses  issues 
pertaining  to  application  of  CFEA,  and  describes^ 
job  aids  used  in  the  CFEA  process. 

APPROACH  TO  CFEA 

A  battalion  Is  a  large  organization  and  one 
would  expect  its  personnel  to  perform  a  large 
number  of  collective  tasks.  A  method  for  delin¬ 
eating  collective,  tasks  is  needed  that  car  offer 
some  assurance  that  a  thorough  listing  of  collec¬ 
tive  tasks  will  be  provided.  Once  the  collective 
tasks  in  a  unit  have  been  specified,  those  per¬ 
formances  must  be  evaluated  to  determine  which 
ones  warrant  the  expenditure  of  training  re¬ 
sources.  An  Important  factor  determining  the 
need  for  training  n  task  is  the  extent  to  which 
deficiencies  in  task  performance  influence  out¬ 
come  of  a  unit  mission.  Thus,  the  method  employed 
to  derive  a  unit’s  collective  tasks  should  allow 


for  determination  of  the  relationship  between  a 
given  task  and  a  unit's  mission. 

An  approach  to  CFEA  has  been  adopted  that 
addresses  both  of  the  above  problems.  The  ap¬ 
proach  to  CFEA  is  described  as  mission-based. 

It  is  described  as  such  because  the  mission  of  the 
unit  to  be  analyzed  is  taken  as  the  point  of  de¬ 
parture  for  the  analysis.  The  CFEA  p^oaess  em¬ 
ploys  techniques  of  systems  analysis^-’ and 
begins  with  specification  of  the  unit  mission(s) 
or  objective (s) ,  proceeds  to  specification  of 
battalion  element  missions,  and  ends  with  delin¬ 
eation  and  description  of  collective  tasks.  The 
process  is  organized  into  two  major  phases:  the 
mission  analysis  and  the  task  analysis.  Figure  1 
presents  the  major  steps  involved  in  CFEA. 

Figure  2  depicts  the  relationship  between  the 
major  products  of  CFEA  and  the  phases  of  the 
analysis.  A  brief  description  of  the  CFEA  process 
is  provided  below.  A  discussion  of  the  advantages 
of  the  mission-based  approach  follows. 
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Figure  2.  Relationship  of  Major  CFEA  Products  and  Phases  of  CFEA 


The  CFFA  Process 


As  indicated  in  Figure  1,  the  mission  analy¬ 
sis  begins  with  development  of  mission  statements 
that  describe  the  objectives  of  the  unit  under 
analysis.  Statements  are  formulated  that  reflect 
the  cap. ib 1 1 i t i os  ot  the  unit's  personnel  and 
equipment  (e.g.,  the  mission  of  an  air  defense 
battalion  would  be  to  provide  low  to  medium  alti¬ 
tude  air  defense)  and  requirements  imposed  upon 
the  unit  by  its  parent  organization  (e.g.,  a  unit 
attached  to  the  Rapid  Deployment  Force  would  have 
the  mission  of  rapid  deployment  via  air  or  sea). 

Once  the  unit's  mission  has  been  established, 
attention  is  turned  to  specifying  the  missions 
performed  by  each  of  the  unit's  organizational 
elements  ns  they  support  accomplishment  of  the 
unit  mission.  Specification  of  element  missions 
is  accomplished  by  determining  the  types  of  func¬ 
tions  that  are  required  to  accomplish  the  unit 
mission  (e.g.,  plan,  direct,  engage,  service, 
repair,  rearm,  refuel,  feed,  etc.),  describing 
the  organizational  structure  of  the  unit,  and 
allocating  the  different  functions  to  organiza¬ 
tional  elements.  For  each  element,  a  statement 
or  series  of  statements  are  prepared  that  de¬ 
scribe  the  specific  nature  of  the  function(s) 
performed  by  the  element. 

After  battalion  clement  missions  have  been 
specified  they  are  assessed  to  determine  their 
criticality.  Criticality  of  a  battalion-element 
mission  is  primarily  a  function  of  three  factors: 
its  direct  effects  on  the  battalion  mission,  its 
effects  on  the  ability  of  the  battalion  to  sus¬ 
tain  operations  and  Its  effects  on  ability  of  the 
battalion  to  survive  on  the  battlefield.  If 
failure  of  a  mission  affects  unit  functioning  in 
any  of  these  areas,  the  mLssLon  is  deemed 
critical . 


When  the  assessment  of  mission  criticality 
is  complete,  the  task  analysis  phase  begins.  This 
phase  consists  of  three  steps.  First,  each  criti¬ 
cal  battalion  element  mission  is  analyzed  to  de¬ 
termine  the  collective  tasks  that  support  it. 
Derivation  of  collective  tasks  is  accomplished  by 
describing  all  of  the  activities  that  are  in¬ 
volved  in  performing  a  given  mission,  grouping 
the  activities  into  units  of  behavior  constituting 
tasks,  and  determining  which  are  performed  by 
collectives  of  personnel. 

Next,  the  collective  tasks  are  assessed  to 
determine  their  criticality  for  training.  A  pri¬ 
mary  factor  affecting  a  task's  criticality  is  its 
influence  or.  the  battalion  element  mission  it 
supports.  Tasks  whose  success  or  failure  affect 
mission  success  are  considered  for  training. 

Tasks  that  are  difficult  or  hazardous  to  perform, 
or  are  performed  infrequently  and  easily  forgotten 
aro  given  high  priority  for  training. 

Finally,  critical  tasks  ore  subjected  to  a 
detailed  analysis  which  .produces  operational  se¬ 
quence  diagrams  (OSDs) .  '  ;  Operational  sequence 
diagrams  display  the  flow  of  task  performances  and 
the  interactions  among  performers.  Task  conditions 
and  standards  are  specified  in  the  same  manner  as 
for  individual,  tasks. 

Advantages  of  the  Mission-Based  Approach 

The  major  products  of  CFEA  are  the  battalion 
mission(s) ,  battalion  element  missions,  and  col¬ 
lective  tasks.  These  products  are  used  to  develop 
Army  Training  and  Evaluation  Plans  (ARTEPs), 
drills,  and  other  training  materials.  Studying 
relationships  between  CFEA  products  depicted  in 
Figure  2,  several  features  of  the  mission-based 


EEST  available  copy 


314 


1  **  ■'“'•'V  HP^f  ^  ;«r  *V 


approach  to  CFKA  arc*  apparent.  First,  develop¬ 
ment  of  mission-oriented  training  is  facilitated 
because  the  tasks  that  support  a  Riven  mission 
are  specified.  Second,  because  the*  starting 
point  of  the  analysis  is  a  battalion  mission,  it 
is  assured  that  all  subsequent  products  are 
mission  relevant.  Third,  breaking  the  battalion 
mission  into  battalion  element  mission.?  prior  to 
specifying  collective  tasks  helps  ensure  a 
thorough  listing  of  tasks.  The  battalion  clement 
missions  provide  a  means  of  subdividing  the  area 
in  which  the  search  for  collective  tasks  occurs, 
bach  battalion  element  mission  focuses  the  task 
analyst’s  attention  in  a  circumscribed  area,  de¬ 
creasing  the  likelihood  that  important  tasks  will 
be  over  Looked,  Finally,  the  mission-based  ap¬ 
proach  to  CFEA  allows  lor  a  more  accurate  assess¬ 
ment  of  task  criticality.  The  relationship  of  a 
particular  task  to  a  battalion  mission  can  be 
traced  through  the  battalion  element  mission  that 
the  task  supports.  A  task's  criticality  can  be 
determined  by  jointly  considering  the  criticality 
of  a  task  to  the  battalion  element  mission  it 
supports  and  the  criticality  of  that  battalion 
element  mission  to  the  battalion  mission. 

Lt  should  be  noted  that  the  traditional 
job-based  approach  to  front-end  analysis  does  not 
otter  the  benefits  of  the  mission-based  approach. 
In  the  Job-based  approach  the  task  analyst  at¬ 
tempts  to  inventory  the  tasks  performed  in  a 
given  job.  Use  of  a  job-based  approach  to  CFEA 
would  result  in  a  thorough  listing  of  tasks  only 
if  the  jobs  in  the  unit  combine  to  fulfill  all  of 
Che  unit's  requirements  for  successful  function¬ 
ing.  This  situation  is  probably  not  the  case 
given  the  fact  that  jobs  in  most  units  have 
evolved  as  a  function  of  equipment  utilized  or 
changes  in  unit  structure  rather  than  on  the 
basis  of  requirements  dictated  by  a  unit's  mis¬ 
sion.  Further,  the  assessment  of  task  critical¬ 
ity  is  more  difficult  in  the  job-based  approach 
because  the  relationship  between  collective  tasks 
and  a  battalion  mission  cannot  be  traced  ex¬ 
plicitly.  Finally,  because  missions  and  the  re¬ 
lationships  between  missions’  anti  tasks  are  not 
specified  in  the  job-based  approach,  mission- 
oriented  training  cannot  be  developed. 

ISSUES  IN  CFKA  APPLICATION 

Ultimately,  the  objective  of  CFEA  is  to  pro¬ 
vide  inputs  for  development  of  training  programs 
that  will  enable  a  unit  to  perform  its  mission. 
There  arc,  however,  issues  regarding  conduct  of 
CFEA  that  impact  the  products  generated  and, 
lienee,  the  training  programs  subsequently  de¬ 
rived.  Review  of  training  materials  that  U9e 
CFEA  products  (e.g.,  ARTEPs)  indicates  there  is 
disagreement  over  what  constitutes  a  collective 
task.  In  addition,  there  is  some  question  as  to 
the  proper  scope  of  CFEA.  Both  of  'hesc  issues 
are  discussed  below. 


Xt'iL-Ng^LX0  r.Preclse  Definition  of  Collective  Task  « 

The  effectiveness  of  training  programs  de¬ 
veloped  using  CFFj\  products  is  partly  dependent 
upon  how  precisely  the  performances  to  be  trained 
are  specified.  In  preparing  collective  task 
statements  users  should  give  careful  consideration 
to  the  definition  of  a  collective  task.  Proper 
collective  task  statements  should  reflect  units  of 
work  that  are  performed  by  two  or  more  people, 
have  a  definite  beginning  and  end,  and  have  pur¬ 
pose  in  and  of  themselves. 

All  too  often,  however,  it  seems  that  collec¬ 
tive  tasks  have  been  confused  with  collections  of 
tasks.  For  example,  in  a  recent  Army  Training  and 
Evaluation  Program  (ARTEP)  one  of  the  collective 
tasks  listed  was  to  perform  periodic  checks. 
Periodic  checks  is  a  class  of  preventive  mainte¬ 
nance  tasks,  some  of  which  are  performed  by  indi¬ 
viduals  and  some  of  which  are  performed  by 
collectives.  Such  task  statements  are  inappro¬ 
priate  for  training  development  purposes  because 
they  are  not  specific  enough  and  must  be  further 
analyzed  in  order  to  determine  what  is  to  be 
trained.  In  an  effort  to  aid  specification  of 
collective  tasks,  a  set  of  criteria  for  evaluating 
performances  is  offered  below. 

When  developing  collective  task  statements 
the  analyst  should  review  each  performance  under 
consideration  and  try  to  specify  its  start  and  end 
points.  If  these  points  can  be  determined,  the 
statement  does  reflect  a  task.  Next,  the  analyst 
should  examine  the  task  to  ensure  that  it  is  in 
fact  a  collective  as  opposed  to  an  individual 
task.  There  arc  several  circumstances  that  give 
rise  to  collective  tasks  and  the  analyst  should 
determine  whether  any  apply  to  the  task  under  con¬ 
sideration.  First,  some  tasks  require  use  of 
skills  found  in  two  or  more  specialties.  Other 
tasks  require  simultaneous  performance  of  task 
steps  in  different  locations^ — Finally,  perform¬ 
ance  of  a  task  can  require  such  a  large  number  of 
skills  that  it  would  be  impossible  for  one  person 
to  perform  the  task  in  a  timely  or  otherwise 
effective  fashion.  Activities  meeting  these  cri¬ 
teria  are  collective  tasks. 

The  Scope  of  CFF.A 

The  stated  objective  of  CFEA  is  to  specify 
and  describe  collective  tasks.  Unfortunately,  if 
the  scope  of  CFEA  is  limited  to  specifying  col¬ 
lective  tasks,  important  individual  tasks  that 
support  accomplishment  of  a  battalion  element 
mission  may  be  overlooked.  The  relationships  be¬ 
tween  missions  and  collective  and  individual 
tasks  are  depicted  in  Figure  2.  Note  that  in 
some  instances  collective  tasks  are  composed  of 
individual  tasks.  In  others,  individual  tasks  do 
not  support  collective  tasks  but  stand  alone  in 
di'rect  support  of  a  battalion  element  mission. 

Many  maintenance,  administrative,  and  supervisory 
tasks,  for  example,  are  performed  by  individuals. 
Therefore,  the  objective  of  CFEA  should  be  to 
specify  all  of  the  performances  (both  collective 
and  individual)  that  support  accomplishment  of  a 
battalion-element  mission.  This  orientation  to 
CFEA  will  enable  development  of  training  programs 
that  support  all  aspects  of  mission  performance. 
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CFKA  JOB  AIDS 


Mission  Function  Classes 


Given  the  size  of  the  units  analyzed  and  the 
massive  amounts  of  data  that  are  generated,  CFEA 
can  seem  overwhelming  to  a  task  analyst.  In  de¬ 
signing  a  process  for  conducting  CFKA,  an  objec¬ 
tive  has  been  to  provide  procedures  that  can  be 
readily  employed  by  Army  training  developers. 
Development  of  procedures  was  based  on  recognition 
of  the  fact  that  front-end  analysis  is  a  somewhat 
subjective  process  that  cannot  be  algorithmized 
to  the  extent  that  the  task  analyst  need  not  make 
decisions  or  judgments.  Given  that,  some  decision 
making  is  required  in  CFKA,  the  concern  was  to 
design  procedures  such  that  the  amount  of  informa¬ 
tion  tc  be  manipulated  by  the  user  at  any  one 
point  in  the  process  Ls  minimized.  This  was 
accomplished  by  breaking  the  process  into  steps 
and  developing  job  aids  whenever  possible.  As  in 
other  training  development  and  evaluation 
processes  such  as. the  Training  Developer's  De¬ 
cision  Aid  (TDDA)  J  and  the  Training  Evaluation 
and  Cost  Effectiveness  Program  (TECEP) ,  '  job 

aids  were  developed  for  CFEA  to  facilitate  organi¬ 
zation  of  data  and  decision  making.  Examples  of 
two  job  aids  used  in  CFKA  are  presented  below. 

As  many  as  ISO  to  200  battalion  element  mis¬ 
sion  statements  might  be  generated  in  a  CFKA. 

The  process  of  developing  this  many  mission 
statements  can  be  time  consuming  and  tedious. 

Also,  because  of  the  large  number  of  element  mis¬ 
sions  performed  in  a  battalion,  it  is  easy  to 
overlook  some.  In  order  to  speed  the  process  of 
specifying  these  missions  and  to  help  ensure 
thorough  coverage,  CFKA  users  are  provided  with 
a  matrix  of  job  aids.  The  job  aid  includes  a 
list  of  generic  functions  which  serve  as  prompts 
for  developing  mission  statements.  One  job  aid 
is  for  specifying  classes  of  mission  functions 
that  apply  to  battalion  elements.  Mission  func¬ 
tion  classes  are  major  groups  of  functions  that 
are  performed  in  support  of  battalion  missions. 

The  three  mission  function  classes  used  in  CFEA 
are  combat  operations,  support,  and  survival. 

T!iis  job  aid,  presented  in  Figure  3,  consists  of 
a  battalion  element  by  function  class  matrix. 

The  user  enters  'X’s  into  the  appropriate  cells 
of  the  matrix  to  indicate  the  function  classes 
that  apply  to  each  element. 


Bn  Elements 

Combat 

Operations 

Support 

- 1 

Survival 

Bn  Hq 

X 

X 

X 

OP/ INTEL 

X 

X 

Fire  Direction 

X 

X 

X 

Admin . 

X 

X 

T  ransportat ion 

X 

X 

Medical 

X 

X 

Motor  Maintenance 

X 

X 

Base  Fire  Platoon 

X 

Figure  3.  Illustration  of  Allocation  of 
Battalion  Elements  to  Mission 
Function  Classes 


Next,  elements  identified  as  performing 
functions  of  a  given  class  are  compared  against 
the  functions  that  make  up  that  class.  Once 
again,  a  matrix  job  aid  is  used  to  identify  and 
record  the  functions  performed  by  each  element. 

An  example  of  such  a  job  aid  used  to  relate 
functions  to  elements  of  an  air  defense  battalion 
is  provided  in  Figure  4.  Finally,  specific 
statements  of  element  missions  are  developed  using 
the  generic  functions  designated  in  the  Figure  4 
job  aid  as  a  point  of  departure. 


Support  Functions 


Bn  Element 

Rearm 

Refuel 

Service 

Repair 

Transport 

Resupply 

Materiel 

Supply  Power 

Fire  Direction 

X 

X 

— 

Bn  Supply 

X 

x 

X 

Motor  Maint. 

X 

X 

X 

X 

System  Support  Pit. 

X 

X 

X 

X 

Transportation 

X 

Figure  4.  Example  of  Allocation  of  Battalion 
Elements  to  Functions  in  a 
Mission  Function  Class 
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°l  C hoso  ■I01’  :iids  Is  important  because  of  the 
manner  Lti  which  they  control  the  user’s  atten¬ 
tion.  The  cells  of  a  given  matrix  force  the  use- 
to  cons  Lifer  in  isolation  each  element  Ln  conjunc¬ 
tion  with  each  function.  Thus,  the  amount  of 
Ln ; or mat. i  on  to  be  considered  by  the  user  at  any 
point  in  the  process  is  minimal  and  the  likeli¬ 
hood  of  a  thorough  listing  of  missions  is  en- 
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ABSTRACT 

Manpower  and  resource  constraints  over  the  next  decade  will  force  a 
reexamination  of  Navy  training  methods.  One  change  that  can  be  predicted 
is  a  shift  in  emphasis  from  maintenance  to  operational  training  as  advanc¬ 
ing  technology  leads  to  more  maintenance-free  systems  with  greatly  enhanced 
functional  capabilities.  To  assist  in  training  approach  trade-offs,  a  bet¬ 
ter  definition  of  the  overall  training  system  is  required  as  well  as  a 
statement  of  training  system  operational  requirements  by  warfare  area.  To 
illustrate,  an  operational  requirement  is  suggested  for  undersea  warfare 
and  the  conclusion  is  drawn  that  increased  emphasis  should  be  placed  on 
organic,  onboard  approaches  to  training  as  substitutes  for  current  shore- 
side  training  methods. 


Training  is  the  single  most  impor¬ 
tant  function  in  which  the  military  ser¬ 
vices  engage  during  peacetime.  To  the 
term  "training,"  should  be  added  the 
phrase  "and  readiness  assessment"  because 
training  by  itself  is  incomplete  unless 
there  are  methods  of  measuring  the  capa¬ 
bilities  of  trained  forces  to  wage  war. 
My  thesis  is  that  within  the  t’avy  in  gen¬ 
eral  but,  more  specifically,  within  the 
field  of  Undersea  Warfare,  the  capabili¬ 
ties  to  train  and  assess  readiness  can  be 
improved. 

That  there  are  problems  is  due  to 
many  factors.  Manpower  availability  of 
the  proper  quantity  and  quality,  the 
characteristics  of  our  weapon  systems, 
the  types  of  tools  available  for  train¬ 
ing,  organizational  attitudes  toward  and 
perceptions  of  the  nature  of  the  problem, 
are  among  those  included.  One  factor 
which  does  not  appear  high  on  the  list  of 
problem  contributors  is  insufficient 
financial  resources.  The  dollars  are 
there.  It  is  how  they  are  applied  that 
can  be  improved. 

There  are  five  thoughts  central  to 
my  argument  which  will  be  expanded  upon 
during  the  balance  of  this  paper.  They 
are: 

n  Constraints  on  manpower  and  re¬ 
sources  will  continue  to  grow  dur¬ 
ing  the  next  decade,  forcing  a 
continuing  reexamination  of  train¬ 
ing  methods. 

•  In  Undersea  Warfare,  the  Training 
System  is  a  "system"  in  only  the 
loosest  sense  of  the  word. 


■  Evolving  technology  will  provide  a 
shift  of  emphasis  from  maintenance 
training,  a  historical  concern,  to 
operator/tactical  training,  the 
principal  problem  of  the  future. 

■  Rational  training  system  trade¬ 
offs  are  not  possible  in  the  ab¬ 
sence  of  a  top  level  statement  of 
Operational  Requirements. 

■  An  Operational  Requirement  pre¬ 
pared  by  persons  knowledgeable  in 
the  Undersea  Warfare  field  will 
lead  to  greatly  increased  emphasis 
upon  organic,  onboard,  training 
approaches. 

We  Americans  can  have  amazingly 
short  memories.  Consider,  for  example, 
popular  attitudes  towards  energy  as  a 
national  problem  between  the  time  of  the 
oil  embargo  of  1973  and  the  glut  of 
1982.  Then  it  was  a  crisis.  Today  we 
hear  l.ttle  of  the  explosive  rhetoric  of 
the  early  70s,  yet  the  basic  problem 
remains.  There  is  an  interesting  paral¬ 
lel  between  the  national  attitudes  toward 
energy  just  noted  and  toward  the  avail¬ 
ability  of  military  manpower.  It  is  im¬ 
portant  that  we  share  a  common  perception 
of  the  manpower  situatior  today  and  of 
future  trends  because  of  the  importance 
of  such  projections  to  the  Navy's  future 
ttairing  system. 

Two  years  ago,  manpower  problems  in 
the  Navy  approached  crisis  proportions. 
Recruiting  goals  were  not  being  met  and 
retention  statistics  were  appalling.  It 
was  not  uncommon  to  read  news  items  about 
ships  which  could  not  operate  because  of 
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personnel  shortages, 
national  attitudes  were 
defense  and  the  Congress 
considerable  dispatch  to 
needed  pay  legislation, 
effects  were  achieved  and 
ebb  was  halted. 


Fortunately , 
strongly  pro- 
reacted  with 
enact  much- 
The  desired 
the  manpower 


I  have  nagging  doubts,  however,  as 
to  whether  the  reversal  is  permanent. 
How  much  has  the  recession  contributed  to 
improvements  in  recruiting  and  reten¬ 
tion?  How  much  is  the  improvement  due  to 
a  national  conservative  swing  to  the 
right,  a  movement  which  appears  to  be 
moderating  as  the  issue  of  military  pro¬ 
gram  vs  social  programs,  guns  vs  butter, 
intensifies?  How  much  confidence  should 
be  placed  upon  historical  data  gathered 
since  implementation  of  the  All-Volunteer 
Force,  data  that  indicate  that  while 
every  pay  raise  has  produced  a  peak  in 
manpower  numbers,  each  subsequent  trough 
between  peaks  has  been  deeper  than  its 
predecessor.  Whether  history  will  repeat 
itself  is  uncertain. 


There  is  little  cause  for  optimism, 
however,  because  there  are  at  least  four 
factors  which  will  tend  to  aggravate  an 
already  shaky  situation.  First,  we  face 
a  shrinking  manpower  pool  of  18-year  olds 
for  at  least  the  next  ten  years.  There 
has  been  enough  written  on  that  subject 
that  further  amplification  should  be  un¬ 
necessary.  Second,  as  the  600-ship  Navy 
goal  is  progressively  reached,  overall 
manpower  requirements  will  increase  even 
further.  Third,  the  trend  toward  more 
sophisticated  weapon  systems  designed  to 
provide  a  sufficient  margin  of  perfor¬ 
mance  advantage  to  offset  Soviet  quanti¬ 
tative  advantages  has  impacted  the  Fleet 
petty  officer  intensity,  that  percentage 
of  enlisted  men  who  are  rated.  An  in¬ 
creased  petty  officer  intensity  tends  to 
translate  into  a  growing  training  pipe¬ 
line.  During  1981,  the  pipeline  absorbed 
21  percent  of  Navy  manpower  and  was  grow¬ 
ing  at  a  rate  of  one  percent  per  year. 
That  21  percent  translated  into  a  2  bil¬ 
lion  dollar  annual  investment.  Lastly, 
accelerating  technological  change  in  the 
civil  sector  is  of  real  concern. 

The  microelectronics  industry  has 
felt  the  impact  of  the  recession  V  ss 
than  other  industry  segments.  It  is  in¬ 
evitable  that  as  the  economy  heats  up, 
the  commercial  electronics,  data  proces¬ 
sing  and  communications  industries  will 
undergo  a  veritable  explosion  and,  in  the 
process,  create  major  demands  for  techni¬ 
cal  manpower.  Such  demands  must  be  sup¬ 
plied  from  that  same  manpower  pool  which 
supplies  Navy  needs. 

Rounding  out  the  earlier  analogy, 
the  critical  manpower  situation  of  two 
years  ago  corresponds  to  the  1973  oil  em¬ 
bargo.  Today  we  may  be  in  that  somewhat 
complacent  period  caused  by  the  apparent 
availability  of  adequate  resources  -  the 
temporary  oil  glut  of  1982.  In  reality. 


however,  if  all  factors  were  considered, 
the  long  term  manpower  picture,  just  as 
in  the  case  of  energy,  looks  pretty  grim. 

Turning  now  to  the  second  of  the 
five  points,  upon  entering  the  Navy,  each 
new  member  becomes  part  of  a  very  large 
and  comprehensive  training  system  which 
affects  every  aspect  of  his  Navy  career. 
This  training  system  can  be  classed  e. 
"system"  in  only  the  loosest  sense,  as 
stated  previously.  It  did  not  derive 
from  the  normal  system  development  pro¬ 
cess  where,  first,  requirements  were 
defined  and  system  functional  character¬ 
istics  were  derived  by  trading  off  alter¬ 
native  solutions  to  the  stated  problem. 
Rather,  the  system  just  grew.  It  evolved 
from  a  variety  of  individual  responses  to 
individual  problems  over  many  years. 
Rarely,  if  ever,  has  there  been  an  at¬ 
tempt  to  analyze  the  entire  system  as  a 
whole.  The  system  is  huge.  It  almost 
defies  analysis  because  of  its  size, 
which  has  been  part  of  the  problem.  How¬ 
ever,  analysis  as  a  system  is  a  "must" 
requirement.  Ever-increasing  demands  for 
support  will  rot  be  satisfied  as  competi¬ 
tion  for  such  resources  grows.  Trade¬ 
offs  between  alternate  solutions  will  be 
necessary.  For  that  purpose,  a  better 
definition  of  the  training  system  and  all 
its  parts  is  required. 

Since  training  impacts  manpower, 
equipment,  and  operation,  it  is  not  sur¬ 
prising  that  the  training  system  has 
grown  to  be  quite  complex  and  that  many 
organizations  have  training-related 
responsibilities.  In  Undersea  Warfare 
alone,  time  does  not  permit  the  listing 
of  all  the  organizations  involved,  but  an 
inspection  of  Figure  1  which  addresses 
only  ASW  and  Atlantic  Fleet  activities 
for  simplicity,  illustrates  some  of  the 
management  complexities  involved.  While 
the  roles  and  contributions  of  tne  orga¬ 
nizations  of  the  Chief  of  Naval  Education 
and  Training  and  of  the  Fleet  Commands 
are  widely  appreciated  and  understood, 
there  are  existing  organizational  ele¬ 
ments  and  expanding  functions  within  the 
Navy  Material  Command  which  have  a  ma";or 
role  in  training  which  may  be  less  well 
known.  Included  in  this  latter  category 
are  those  organizations  responsible  for 
underwater  ranges,  targets  and  for  train¬ 
ing  capabilities  embedded  »n  weapon  sys¬ 
tems  under  development. 

The  Atlantic  Undersea  Test  and  Eval¬ 
uation  Center  may  have  had  its  origins  in 
the  development,  test,  and  evaluation  of 
underwater  weapon  systems  but  its  contri¬ 
bution  to  training  and  readiness  assess¬ 
ment  through  the  monitoring  of  SHAREMS 
and  MiniWars  and  in  SSN/SSBN  PCO  training 
has  grown  significantly.  Also,  the  scar¬ 
city  of  submarines  for  target  services 
has  placed  increased  emphasis  and  dic¬ 
tated  growing  resource  requirements  for 
such  tools  as  the  Expendable  Mobile  Acous¬ 
tic  Training  Target,  the  target  Mk  30  and 
its  potential  successor,  the  Advanced  ASW 
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Figure  1.  ASW  Training  -  Interested  Organizat ions 


Target.  Lastly,  evolving  silicon  tech¬ 
nology,  and  in  particular  microproces¬ 
sor/microcomputer  families  coupled  with 
low  cost  memory,  has  made  it  possible  to 
either  add  onto  or  to  embed  training 
tools  in  systems  already  in  the  Fleet  or 
entering  system  development.  In  the  for¬ 
mer  category  falls  the  AN/BQR-T4  in¬ 
stalled  in  the  Poseidon-class  SSBNs,  and 
the  AN/SQS-T5  slated  for  the  Aegis-class 
cruisers.  In  the  latter  category  are 
found  *-he  embedded  target  stimulators  of 
the  Submarine  Active  Detection  System  now 
in  engineering  development  and  the 
planned  organic  trainers  for  the  DD-963 
and  FFG-7  class  AN/SQQ-89s  and  for  the 
SSN-688s  beginning  with  SubACS  Phase  0  in 
SSN-751 . 

All  of  the  above  activities  either 
complement  or  duplicate  the  training 
functions  normally  falling  under  the  aus¬ 
pices  of  the  Chief  of  Naval  Education  and 
Training.  While  there  may  not  appear  to 
be  competition  for  resources  because  they 
may  be  funded  from  different  line  items, 
realistically  their  contr  ibut io.'.s  to 
training  and  readiness  should  be  measured 
by  a  common  yardstick.  Such  assessment 


will  continue  to  be  very  difficult  until 
a  better  system  model  of  the  overall 
training  system  can  be  formulated. 

As  I  stated  in  my  third  introduction 
point,  historical  needs  for  maintenance 
training  will  moderate  and  be  replaced  by 
increased  requirements  for  training  in 
operations  and  tactics.  Systems  deri¬ 
sions  have  already  been  made  that  will 
reduce  the  need  for  maintenance  train¬ 
ing.  This  fact  may  be  difficult  to 
a:cept  by  an  operator  in  the  Fleet  today 
saddled  with  the  difficult  task  of  keep¬ 
ing  an  older  equipment  on  line,  one  which 
is  neither  modular  in  design  nor  equipped 
with  built-in  fault  detecting  and  local¬ 
izing  capabilities.  As  t«iC  never  gener¬ 
ations  of  equipments  are  being  fielded, 
however,  maintenance  actions  consist  of 
plug-in  replacement  cf  modules  in  re¬ 
sponse  to  system-generated  instructions. 
For  systems  such  as  SubACS  which  will 
appear  in  the  Fleet  in  the  latter  half  of 
the  decade,  system  design  requirements 
specify  a  sixty-day  maintenance-free  mis¬ 
sion,  thus  eliminating  ever,  the  simple 
module  replacement  maintenance  actions 
roted  above. 
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That  these  changes  are  fortuitous  is 
an  understatement.  With  the  manpower 
projections  cited  earlier,  the  Navy  would 
be  unable  to  man  its  ships,  if  prior 
maintenance  design  practices  were  carried 
forward  to  systems  of  the  future.  The 
day  of  the  electronics  "super  tech"  who 
could  make  onboard  module  repair  and  jury- 
rig  a  failed  svetem  into  some  form  of 
operation  is  disappearing,  if  not  already 
gone.  Advancing  electronic  device  tech¬ 
nology  leading  to  dramatic  increases  in 
functional  complexities  and  densities  has 
moved  the  field  beyond  the  levels  that 
can  be  taught  in  A,  B,  or  C  schools. 
There  may  be  super  techs  in  the  future 
but  they  will  be  found  at  Intermediate 
Maintenance  Activities  (IMA)  and  will 
probably  be  called  "engineers". 

Rather  than  bemoan  such  changes  as 
some  may,  we  should  be  thankful.  The 
positive  growth  rate  of  the  training 
pipeline  over  the  past  years  should  slow 
and  eventually  reverse.  I  argue  that 
even  now  there  are  sonarman  A-school 
graduates  who  have  no  need  for  a  ten-week 
course  in  basic  electricity  and  electron¬ 
ics  for  the  systems  to  which  they  will  be 
assigned.  Why  teacn  it,  if  it  cannot  be 
employed?  Do  we  realty  need  to  teach 
soldering  techniques?  I  shudder  to  think 
of  the  damage  that  a  misguided  soldering 
gun  can  produce  to  a  multi-layer,  high 
density  standard  electronic  module,  or  to 
a  power  supply  or  wiring  backplane.  I  am 
convinced  that  maintenance  training 
should  be  confined  in  the  very  near 
future  to  the  reading  of  fault  indicators 
and  simple  module  plug-in  substitution 
for  repair. 

While  maintenance  training  require¬ 
ments  may  decrease,  the  need  for 
operational/tactical  training  and  for 
readiness  assessment  is  increasing  sig¬ 
nificantly.  The  Navy’s  response  to  quan¬ 
titative  inferiority  in  submarine  num¬ 
bers,  for  example,  has  been  to  maintain  a 
significant  qualitative  performance  supe¬ 
riority.  ASW  combat  teams  of  fewer  men 
are  controlling  systems  with  more  func¬ 
tional  modes,  weapon  mixes  of  greater 
variety  and  larger  action  areas  than 
would  have  been  thought  possible  just  a 
few  years  ago.  Towed  arrays,  Lamps  1 1  i , 
Harpoon,  Tomahawk,  over-the-horizon  tar¬ 
geting  are  just  a  few  ot  the  many  new 
tools  whose  efficient  application  our 
operators  of  the  80s  will  need  to  mas¬ 
ter.  The  training  burden  will  be  large. 

Up  to  this  point,  I  have  sketched  a 
picture  of  increasing  manpower  problem.; 
with  a  shift  in  training  requirements  em¬ 
phasis  from  maintenance  to  operations  and 
tactics.  Also,  l  have  stated  that  the 
training  system  which  musr  take  scarce 
manpower  resources  and  transform  them 
into  war-fighting  capable  teams  is  not 
completely  modelled.  This  makes  trade¬ 
off  assessments  of  competing  proposed 


modifications  to  the  system  quite  diffi¬ 
cult.  My  fourth  point  is  as  follows:  In 
order  to  clarify  the  training  system 
model  and  to  assist  in  the  important 
resource  allocation  trade-off  process, 
top-level  training  system  operational 
requirements,  by  warfare  area,  are 
needed.  Let  me  illustrate  what  I  mean  by 
"top-level"  by  a  suggested  version  of  an 
operational  requirement  beginning  with  a 
statement  of  objective. 


^ _ 1 

viduals 

and 

as  teams,  to  perform  effec- 

tively 

in  the  operating  environment  and 

to  measure 

combat  readiness  of  both 

equipment,  ana  personnel.  nac.n  warfare 
area  has  unique  requirements  which  must 
be  satisfied  if  the  overall  objective  is 
to  be  met.  In  undersea  warfare  the  key 
requirement  is  to  teach  interpretative 
analysis  skills  as  well  as  operating  pro- 
cedu  r es . 


There  is  a  significant  difference 
between  knowing  how  to  operate  equipment, 
ships,  and  forces  and  how  to  use  such 
resources.  to  achieve  an  operational 
objective.  Most  training  systems  concen¬ 
trate  on  the  procedural  aspects  of  opera¬ 
tions  training  and  neglect  the  more  dif¬ 
ficult  training  challenge  which  is  how  to 
interpret  and  evaluate  the  contribution 
of  each  element  of  the  system  to  the 
solution  of  the  overall  problem.  Inter¬ 
pretative  skills  weigh  heavi ly  in  under¬ 
sea  warfare  in  contrast,  for  example,  to 
surface  ship  anti-air  warfare. 

The  highly  complex  underwater  acous¬ 
tic  environment,  the  low  speed  of  sound 
in  contrast  to  that  of  electromagnetic 
energy,  and  the  low  speeds  of  acoustic 
targets  of  interest  relative  to  thase  ot 
aircraft  and  missiles  result  in  tactical 
problems  which  evoive  over  minutes  and 
hours  rather  than  in  seconds.  The  pro¬ 
cesses  of  detection,  classification, 
localization  and  weapon  control  are  very 
dependent  upon  team  skills  in  analysis 
and  interpretation  of  data  which  are  not 
provided  with  the  precision,  uniqueness, 
anl  rapidity  of  radar.  In  contrast,  the 
speed  with  which  the  AAW  problem  develops 
with  potentially  threatening  targets  mea¬ 
suring  in  the  hundreds  at  any  point  in 
time,  points  to  system  operation  which  is 
procedure  dominated.  Threat  assessment 
algorithms  and  rules  of  engagement  must 
be  predetermined.  There  just  is  not  rime 
for  extensive  .nterpretat ion  and  analy¬ 
sis,  for  the  reorientation  of  forces  or 
the  placement  of  new  sensors  nor  are  such 
steps  needed.  I  do  not  mean  to  infer 
that  procedural  training  is  unnecessary- 
in  undersea  warfare.  To  the  contrary, 
procedural  knowledge  is  very  important. 
Without  training  in  equipment  operation, 
communication  protocol,  and  tactical  con¬ 
straints,  as  examples,  a  single  platform 
ASW  team  or  a  multi-platform  task  group 
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cannot  begin  to  function.  In  fact,  the 
knowledge  of  proper  operational  proce¬ 
dures  must  become  second  nature.  Too 
often,  however,  training  in  undersea  war¬ 
fare  stops  at  procedural  training  (the 
tip  of  the  iceberg  of  the  overall  under¬ 
sea  warfare  training  problem)  and  ignores 
the  far  more  difficult  aspects  which  are 
the  teaching  of  interpretative  and  analy¬ 
sis  skills. 

The  second  most  important  require¬ 
ment  is  to  provide  rapid,  accurate  assess¬ 
ment  of  system  and _ team  performance  fol¬ 

lowing  a  training  event.  This  require¬ 
ment  is  not  unique  to  undersea  warfare 
and  should  be  a  characteristic  of  our 
training  systems,  independent  of  warfare 
area.  Training  without  prompt,  accurate 
feedback  may  do  more  harm  than  good  by 
developing  a  false  sense  of  accomplish¬ 
ment  on  the  part  of  participants  when,  in 
fact,  the  objective  of  the  exercise  is 
not  accomplished.  Again  procedural 
training  is  relatively  straightforward, 
and  performance  assessment  can  be  accu¬ 
rate  and  rapid  with  proper  monitoring. 
Overall  performance  assessment  involves 
determining  whether  proper  interpreta¬ 
tions  of  available  data  were  made  and 
appropriate  actions  initiated.  ?h;s  is 
far  more  diffi  ult.  It  requires  i  truth 
table  (an  accurate  navigational  plot  of 
the  positions  of  all  platforms,  sensors, 
targets  and  weapons)  and  a  means  of  com¬ 
paring  this  truth  table  with  what  the 
exercise  participants  thought  had  oc¬ 
curred. 

In  keeping  with  the  earlier  state¬ 
ment  that  maintenance  training  require¬ 
ments  are  decreasing  and  will  continue  to 
do  so  in  the  future,  the  training  system 
should  teach  only  the  minimum  maintenance 
skills  consistent  with  organizational 
level  support  requirements. 

With  the  steady  growth  in  operating 
demands  upon  our  submarine  forces,  it  is 
not  surprising  that  to  make  minimum 
demands  on  submarine  services  as  target s 
should  be  a  pr ior i ty  requirement.  While 
it  is  unrealistic  to  believe  that  the 
need  for  submarine  services  for  “xerciso 
targets  can  be  eliminated,  alternate 
approaches  to  employing  operating  subma¬ 
rines  as  targets  must  be  developed  if  wo 
are  to  maintain  a  reasonable  state  of 
readiness  in  ASW. 

Experience  has  also  shown  that  the 
provision  of  frequent  training  opportuni- 
t  les  is  ar7'  important  requirement.  A 
trained  operator  or  team  must  practice 
regularly  to  keep  developed  skills.  This 
has  been  deronstrated  by  tests  of  SSBN 
sonar  operators  during  operating  patrols 
and  probably  applies  to  other  members  of 
the  ASW  team.  Since  funu.ng  and  other 
constraints  will  probably  continue  to 
limit  ship  underway  and  aircraft  flight 
time,  it  is  imperative  to  use  every  op¬ 
portunity  to  exercise  and  train.  The 
meaning  of  the  term  “frequent"  should 


depend  on  the  perishability  of  the  devel¬ 
oped  skills  as  well  as  measured  perfor¬ 
mance  on  previous  exercises.  It  defi¬ 
nitely  is  not  the  one  or  two-year  gap 
often  experienced  by  frigates  in  conduct¬ 
ing  a  multi-platform  SHAREM  or  MiniWar 
nor  the  quarterly  opportunity  VP  air 
crews  receive  to  drop  one  actual  torpedo. 

Another  requirement  is  that  the 
training  system  be  transparent  to  the 
operator,  that  is,  is  indistinguishable 
from  a  true  operating  system.  This 
requTrement  can  be  interpreted  two  ways. 
First  that  any  simulated/stimulated  tar¬ 
gets  or  displayed  information  have  per¬ 
fect  fidelity  with  the  real  world  and, 
second,  that  in  addition,  operators  are 
not  alerted  and  do  not  suspect  that  an 
exercise  is  in  progress.  The  former  is 
considered  of  principal  importance  when 
the  training  operation  is  striving  to 
develop  evaluative  and  interpretive 
skills.  While  there  are  occasions  when 
it  ;s  desirable  to  test  the  state  of 
operator  alertness,  that  testing  is  rated 
considerably  below  the  requirement  for 
realism.  Realism  also  implies  that 
training  exercises  avoid  stereotyping  to 
where  tear,  participants  recognize  a  repe¬ 
tition  of  a  former  event. 

It  is  also  clear  that  the  capabi 1 i ty 
to  train  at  3ll  leve Is  of  the  team  --  the 
individual  sonar  c perator,  the  sonar 
team,  the  combat  system  team  is  a  re¬ 
quirement.  Any  significant  investment  in 
a  training  system  must  support  all  levels 
of  team  participants  --  individuals  as 
well  as  groups. 

Another  desirable  attribute  is  to 
impose  no  geographic  constraints.  Geo¬ 
graphic  Independence  is  important.  To 
iimit  the  area  constrains  accessibility 
to  training  and  may  imply  a  restrictive 
range  of  ocean  environmental  conditions. 

A  problem  that  has  occurred  in  past 
training  systems  is  performance  inconsis¬ 
tencies  between  the  equipment  used  for 
training  and  that  employed  in  Fleet  use. 
It  is  an  important  requirement  that  con¬ 
sistency  between  equipmen  t  used  fo*:  Fie  at 
operation,  and  that  upon  whTch  training 
occurs  be  provided.  Di7Terences  can 
result  from  unintended  breakdov;ns  in  sys¬ 
tem  configuration  management  during 
equipment  evolution  and  development  or 
from  deliberate  efforts  to  save  cost 
through  the  use  of  inexact  simulations  of 
actual  system  hardware.  Such  differences 
may  detract  from  the  effectiveness  of 
training  exercises.  What  is  more  impor¬ 
tant,  however,  they  could  constitute  dis¬ 
tractions  during  time  of  stress  during 
actual  operations. 

Nine  requirements  statements  have 
beer,  listed.  There  may  be  others  which 
experienced  personnel  in  the  field  can 
add.  The  list  may  require  modification. 
The  specific  details  are  not  as  important 
as  the  existence  of  some  formal  statement 
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of  requirements  to  facilitate  system 
trade-offs  and  the  resource  allocation 
management  process. 

The  statement  of  requirements  just 
developed  provides  positive  support  for 
the  fifth  point  of  my  argument  that  or¬ 
ganic,  onboard,  training  approaches 
should  receive  increased  emphasis.  Con¬ 
sider  with  me  how  each  of  the  operational 
requirements  just  listed  can  be  satisfied 
most  logically  by  onboard  training  ap¬ 
proaches  and  the  attendant,  very  real 
benefits  which  can  accrue. 

Requirement  1  -  Interpretative  and 

Procedural  Skills.  To  properly  develop 
interpretative  analysis  skills  requires 
environment  models  which  faithfully  rep¬ 
resent  all  of  the  complexities  and  infi¬ 
nite  variations  .in  ocean  acoustic  propa¬ 
gation  and  in  background  noise  and  rever¬ 
beration  from  both  natural  and  manmade 
sources.  The  required  degree  of  realism 
can  undoubtedly  be  created  artificially 

but  at  tremendous  expense  for  development 

and  operation.  Normally  the  latter  im¬ 

plies  limited  accessibility  because  only 
a  few  complex  and  expensive  systems  can. 
be  afforded.  The  counter  to  the  expen¬ 
sive  ocean  simulator  is  to  use  the  ocean 
itself  by  mixing  artificial^  generated 
threat  data  with  t-ne  normal  information 
detected  by  ship's  sensors. 

Procedural  training,  on  the  other 

hand,  does  not  require  exact  realism  in 
environment  modelling.  Simple  models  can 
be  embedded  economically  in  sensor  sys¬ 
tems  to  support  procedural  training  in 
port.  Some  may  suggest  thut  portable 
V3n-instal led  systems  located  in  major 
fleet  centers  car  provide  the  function 
described.  Procedural  training  in  port 
could  be  satisfied  by  such  an  approach 
although  front-end  target  injection  by  a 
portable  system  may  be  a  poor  choice, 
because  of  the  susceptibility  to  electri¬ 
cal  interference  of  the  early  processing 
stages  of  sonar  sensors.  The  van  sys¬ 
tems,  however,  will  fall  short  in  realism 
for  interpretative  training  ncr  will  they 
satisfy  the  need:;  of  ships  on  extended 
deployments  such  as  those  in  the  Indian 
Ocean  who  now  are  suffering  from  a  1  acfc 
of  ASW  traininq  opportunities. 

Requirement  2  -  A  Truth  Table.  A 

principal  limitation  from  which  onboard 
training  efforts  have  always  suffered  has 
been  the  inability  to  provide  rapid,  ac¬ 
curate  feedback  on  ship  performance  fol¬ 
lowing  a  training  exercise.  The  greatly 
expanded  use  of  range  facilities  such  as 
AUTEC  has  been  motivated  by  this  require¬ 
ment.  While  the  ranges  have  the  ability 
to  accurately  measure  performance,  feed¬ 
back  has  not  always  been  rapid.  “Hot 
Washupa*  and  “Quick  Look-  reports  de¬ 
signed  to  quickly  disseminate  information 
may  have  benefited  Command  levels  but 
have  not  been  sufficiently  detailed  nor 
prompt  to  be  of  value  to  the  working 
level  members  of  the  ASW  team. 


The  Post-Operational  Analysis  and 
Exercise  Review  Program,  acronym  PACER, 
developed  for  use  initially  on  the 
BARSTUR  Hawaiian  Range,  was  designed  to 
correct  the  above  deficiency.  PACER  pro¬ 
vides  a  critique  to  participating  units 
within  three  days  of  an  exercise  which 
consists  of  a  narrated,  pre-edited  and 
time-compressed  video  replay  of  3-D  range 
tracking  data  merged  with  shipboard  tar¬ 
get  motion  analysis  data.  The  program 
has  been  very  effective  and  similar  ef¬ 
forts  are  being  applied  at  other  ranges. 

As  valuable  as  current  instrumented 
ranges  and  exercise  feedback  systems  are, 
they  do  suffer  from  limited  accessibil¬ 
ity.  The  average  ship  might  expect  an 
opportunity  only  once  a  year  to  exercise 
on  rangr.  In  contrast,  an  organic 
trainer  with  embedded  target  injection 
capabilities  provides  the  same  benefits 
for  target  detection,  classification, 
target  motion  analysis  and  weapon  direc¬ 
tion  functions  as  an  instrumented  range 
and  a  PACER  system  without  the  availabil¬ 
ity  restrictions.  The  only  function  not 
covered  is  live  weapon  firing.  Torpedo 
weapon  firings  are  not  normally  conducted 
in  the  open  ocean  because  of  the  need  for 
retriever  services  and  most  firings  occur 
on  the  instrumented  ranges.  If,  however, 
retriever  services  are  available,  there 
are  techniques  which  support  weapon  fir¬ 
ing  in  an  open  ocean  exercise  while  em¬ 
ploying  an  organic  trainer. 

Requirement  3  -  Maintenance  Skills, 
With  maintenance  at  the  organizational 
level  confined  to  module  substitution, 
onboard  training  is  practical.  A  variety 
of  new  *-ools  such  as  the  interactive 
video  disk  can  provide  training  support. 
A  video  disk/sonar  display  marriage  would 
permit  individual  operator  maintenance 
traininq  di/ing  non-battle  conditions 
when  underway  or  in  port  when  manning  of 
all  consoles  for  routine  operations  is 
not  a  requirement.  The  interactive  video 
disk  coupled  to  system  sonar  displays 
should  also  eliminate  maintenance  manuals 
as  we  know  them  today. 

Requirement  4  -  Minimize  Submarine 

Service  Demands.  An  onboard  training 
system  based  upon  artificial  target  in¬ 
jection  is  obviously  in  keeping  with  this 
requirement.  There  are,  of  course,  al¬ 
ternate  approaches  possible  such  as 
expendable  or  reusable  mobile  training 
targets.  All  existing  and  planned  tar¬ 
gets  such  as  the  Mk  38,  EMATT,  the  Mk  33 
or  the  Advanced  ASW  Target  suffer  as  com¬ 
bat  system  training  targets  because  of 
the  technical  difficulty  of  achieving 
acoustic  fidelity  or  tactical  realism. 
In  order  to  provide  some  form  of  target 
azimut?  1  extent,  for  example,  it  is  nec¬ 
essary  tow  multiple  transponders  to 

produce  L  ihlighcs.  Towed  arrays  compli¬ 
cate  syu  "  design  as  well  as  che  launch¬ 
ing  and  retrieval  process  while  still 
providing  less  than  perfect  target 
realism.  The  faithful  reproduction  of 
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submarine  low  frequency  radiated  tonals 
is  also  quite  difficult  within  transducer 
rize  constraints  which  are  practical  for 
expendable  or  torpedo-sized  targets.  The 
speed  and  depth  characteristics  of  cur¬ 
rent  threat  targets  are  also  a  challenge 
to  emulate.  Oi  board  target  stimulators 
suffer  from  none  of  the  above  limita¬ 
tions.  High  fidelity  target  signatures 
and  realistic  target  motion  dynamics  have 
been  demonstrated  repeatedly. 

Requirement  5  -  Frequent  Training 
Oppoi tuni ties.  Organic  trainers  usable 
underway  and  in  port  certainly  satisfy 
this  requirement  better  than  centralized 
facilities  which  require  transport  of 
teams  to  training  sites.  In  addition, 
ensuring  the  availability  of  all  appro¬ 
priate  members  of  a  team  to  be  trained 
can  be  a  very  real  challenge  because  of 
conflicting  personnel  requirements  when  a 
ship  is  in  port. 

Requirement  6  -  Training  System 
Transparency .  As  was  noted  previously, 
this  requirement  can  be  interpreted  in 
two  ways.  First,  there  is  the  require¬ 
ment  for  high  fidelity  in  order  that 
operators  in  training  to  detect  and  clas¬ 
sify  underwater  targets  not  do  so  based 
uupon  some  misleading  artifact  of  the 
simulation  which  would  not  appear  in  the 
real  world.  Considerable  development 
effort  has  been  applied  to  this  area  for 
training  systems  now  in  use  by  the  Navy's 
strategic  -  miss i le -  f ir i ng  submarines.  A 
review  board  consisting  of  members  of  the 
intelligence  community  and  submarine 
school  instructors  exhaustively  test  all 
target  simulations  prior  to  their  release 
to  the  'leet  to  ensure  real  world  fidel¬ 
ity.  This  requirement  applies  equally  to 
either  shoreside  or  onboard  training  sys¬ 
tems. 

The  second  interpretation  concerns 
operator/team  alertness.  Conventional 
wisdom  in  the  past  has  precluded  the  use 
of  target  injection  systems  to  determine 
the  degree  of  alertness,  of  sensor  opera¬ 
tions.  The  general  concern  has  hee.i  that 
exposing  the  inadequacy  of  an  operator  by 
injecting  a  target  without  his  knowledge 
would  lower  moiale  and  lead  to  probable 
injection  system  sabotage.  That  view  is 
questionable.  It  is  reasonable  to  be¬ 
lieve  however,  that  an  operator  must  have 
a  reasonable  probability  of  winning  in 
any  contest  or  otherwise  morale  will  be 
adversely  affected.  Because  of  the  total 
iuabequacy  of  tools  to  support  opera¬ 
tional  skills  development  today,  the 
probability  of  winning  is  low.  However, 
or.  some  SSBNs  with  organic  stimulator 
training  capability,  the  operators  have 
developed  enough  confidence  in  their 
capabilities  that  on  their  own  initiative 
they  have  turned  the  learning  process 
into  a  game  where  they  pit  their  detec¬ 
tion  and  classification  skills  against 
one  another. 


Requirement  7  -  Training  at  All  Team 
Levels .  Training  all  members  of  the  ASW 
teams  includes  training  extending  from 
sensor  operators  up  to  and  including  the 
command  level.  For  a  specific  ship  in  a 
specific  exercise,  the  extent  of  specific 
command  involvement  is  always  a  function 
of  the  judgement  of  the  Commanding  Offi¬ 
cer  in  terms  of  the  specific  objectives 
of  the  evolution  and  the  needs  of  the  ASW 
team.  However,  the  training  system 
should  permit  the  Commanding  Officer  to 
e^sreise  his  tactical  judgements  and  mea¬ 
sure  the  results  against  exercise  cri¬ 
teria  ar  a  part  of  the  ship's  total  prep- 
aracion  for  war.  For  those  ASW  surface 
ships  being  equipped  with  tactical  ASW 
sensors  only  recently  in  the  inventory, 
the  capability  for  the  Commanding  Officer 
of  these  ships  to  develop  and  evaluate 
new  interpretive  ana  tactical  skills  is 
an  urgent  necessity. 

Conflicting  personnel  requirements 
when  a  ship  is  in  port  were  cited  av-  a 
limitation  of  shoreside  training  sysezms 
when  discussing  the  requirement  for  fre¬ 
quent  training  opportunities.  These  same 
limitations  apply  when  attempting  to 
train  the  entire  team  ashore,  especially 
at  the  senior  team  member  level. 

An  organic,  onboard  training  capa¬ 
bility  with  inherent  "truth  table"  fea¬ 
tures  would  certainly  help  in  this  area. 
A  ship  team  from  Commanding  Officer  on 
down  could  stage  ASW  engagements  as  fre¬ 
quently  as  desired  without  reliance  upon 
outside  services  and  be  free  to  make  all 
the  typical  mistakes  inherent  in  the 
learning  process  without  outside  observa¬ 
tion  and  command  embarrassment.  Once 
individual  ship  confidence  in  team  abili¬ 
ties  is  reached,  it  is  not  difficult  to 
envision  training  operations  between 
pairs  of  ships  using  stimulator  targets, 
between  ships  and  aircraft,  and  eventu¬ 
ally  among  complete  ASW  Task  Units. 


Requirement  3  -  N^-  Geog raphi  :  Con- 
straints .  Fixed  ranges  such  as  AUTEC, 
BARSfUR  and  St.  Croix  impose  geographic 
constraints  which  not  only  limit  accessi¬ 
bility  but  constrain  training  to  the  en¬ 
vironmental  conditions  at  that  particular 
location.  Such  conditions  may  not  be 
typical  of  the  broad  spectrum  of  condi¬ 
tions  experienced  throughout  the  world. 
AUTEC  is  a  good  illustration  where  the 
"bathtub"  bathymetry  of  The  Tongue  of  tne 
Ocean  precludes  effective  use  of  low  fre¬ 
quency  active  sonar  because  of  very  high 
reverberation  interference.  Organic 
trainers  do  not  constrain  training  opera¬ 
tions  to  a  particular  area. 

Requirement  9  -  Consistency  Between 

T  tamino _ and  Cpe'r'aHonal  EquipmentT  ft 

Ts  obvious  that  onboard  trainers  which 
stimulate  operational  equipment  ensure 
such  consistency. 
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In  summary,  the  material  that  I  have 
oresented  in  thi  3  paper  suggests  changes 
to  the  manner  in  which  ASW  training  is 
conducted  in  the  Navy  today.  My  conten¬ 
tion  is  that  manpower  and  resource  con¬ 
straints,  coupled  with  the  nature  of  the 
submarine  threat,  demand  that  the  Navy 
extract  every  ounce  of  capability  inher¬ 
ent  in  our  sophisticated  ASW  systems. 
Accordingly,  training  and  readiness 
assessment  are  critical.  Fortunately, 
technology  advances  will  permit  heavier 
concentration  on  operator  training.  How¬ 
ever,  logical  and  necessary  changes  are 
being  impeded  bv  unclear  perceptions  of 
our  current  training  systems  and  the  lack 
of  a  concise  statement  of  top  -  level 
requirements.  I  have  suggested  such  a 
statement  of  requirements  and  argued  that 
orqanic,  onboard  training  systems  come 
closest  to  satisfying  them. 

I  am  encouraged  that  others  recog¬ 
nize  the  importance  of  organic  training 
approaches  as  a  means  of  shortening  the 
training  pipeline  and  increasing  the  ef¬ 
fectiveness  of  personnel  utilization. 
The  Enlisted  Personnel  Individualized 
Career  System  (EPICS)  is  a  case  in  point 
which  is  being  applied  to  the  NATO  Sea 
Sparrow  program.  The  goal  of  that  effort 
is  to  develop  skill  level  III  trained 
personnel  at  the  end  of  six  years  while 
having  devoted  only  50  weeks  to  shoreside 
schooling,  including  recruit  training. 
If  successful,  this  change  will  consti¬ 
tute  a  significant  improvement  in  utiliz¬ 
ation  and  resource  application. 

Underlying  all  that  I  have  said  in 
stressing  the  need  for  reducing  the 
training  pipeline  and  in  arguing  for  sup¬ 
port  of  onboard  training,  is  a  fundamen¬ 
tal  belief  that  everyone  enjoys  doing 


that  which  he  knows  he  does  well.  We  all 
learned  that  lesson  early  in  life  in 
school  where  the  subjects  in  which  we 
received  good  grades  were  normally  our 
favorites.  Lack  of  confidence  in  one's 
ability  to  perform  assigned  tasks  or, 
what  is  worse,  confusion  as  to  what  one's 
job  really  is,  is  self-defeating  and  a 
morale  depressant.  I  am  convinced  that 
training  can  be  fun,  that  well-trained 
Navy  personnel  enjoy  high  self-esteem, 
pride  of  organization,  good  morale  and, 
accordingly,  will  tend  to  remain  in  the 
service.  Adopting  the  suggestions  pro¬ 
posed  in  this  paper  can  help  to  foster 
that  happy  situation. 
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ABSTRACT 

Since  1973,  General  Physics  Corporation  has  been  developing  operational  training  program 
materials  for  use  on  board  surface  ships,  submarines,  and  aircraft  of  the  United  States 
Navy.  These  programs  have  been  oriented  toward  providing  an  individual  ship  or  squadron  the 
capability  to  increase  its  operational  performance  in  a  particular  warfare  area  through  a 
series  of  scenario-based  exercises,  utilizinq  the  unit's  own  inherent  training  assets  and 
capabilities.  These  programs  have  been  evolutionary  and  as  the  programs  have  matured,  many 
problems  have  arisen  which  have  complicated  program  organization,  development.  Fleet 
implementation,  and  overall  use  of  the  materials.  These  problems  include:  confusion  and 
lack  of  understanding  of  the  nature  of  onboard  training;  lack  of  documentation  for  combat 
systems;  conflicting  or  non-existent  operational  guidance;  limitations  of  imbedded  training 
modes  and  systems;  inability  to  keep  pace  with  system  upgrades  and  modifications; 
difficulties  in  implementing  programs  and  materials;  and  difficulties  in  sustaining  interest 
and  use  ir.  Fleet  units.  Each  of  these  problems  has  .been  encountered  and  dealt  with,  with 
varying  degrees  of  success.  These ^lessons  learned4  and  other  recommendations  are  presented, 
for  the  consideration  of  others  embarking  on  development  of  onboard  training  efforts.  Navyjv 
policy  toward  onboard  training  and  funding  are  discussed  and  recommendations  made.  rinally  a 
list  of  thumbrules  for  onboard  training  program  and  material  developers  is  presents. 


INTRODUCTION 

In  1973  General  Physics  Corporation 
commenced  development  of  methodology  and 
materials  to  support  onboard  operational 
training  and  individual  and  team  assessment  in 
sonar  and  fire  control /attack  center  operations 
aboard  U.S.  attack  submarines.  This  initial 
effort,  Sponsored  and  directed  by  Commander, 
Submarine  Force,  U.S.  Atlantic  Fleet,  was 
designed  to  formalize  and  standardize  onboard 
combat  system  training,  as  well  as  to  provide  a 
submarine  commanding  officer  or  his  operational 
commander  with  a  quant itat i ve,  objective 
assessment  of  the  ship's  overall  operational 
readiness  in  combat  system  operation.  The 
Submarine  Sonar  and  Fire  Control  Operational 
Readiness  Assessment  and  Training  Program 
(eventually  shortened  to  Submarine  Operational 
Readiness  Assessment  and  Training  (SORAT) 
Program)  was  developed  in  close  cooperation  with 
the  operating  Fleet  and  addressed  operation  of 
current  onboard  systems  usinq  ex.itinq  tactics 
and  procedures,  thus  encompassing  the  day-to-day 
training  requirements  of  an  operating  ship.  A 
key  feature  of  the  program  was  the  extensive  use 
of  existing  equipment  and  imbedded  training 
modes  or  capabilities  of  the  combat  system. 

SORAT  was  mass  produced  and  distributed  to 
all  Fleet  submarine  units  (including  versions 
for  SS  and  SSBN)  in  1978.  About  this  time  the 
program  was  adapted  to  support  the  rapid 
conversion  of  submarines  to  the  digital  AN/BQQ-5 
Sonar  and  the  MK  117  Fire  Control  System. 

Today,  the  program  exists  in  13  different 
version*  (tailored  to  various  sonar/fire  control 
suites)  and  is  aboard  virtually  every  U.S. 
submarine. 

The  same  methodologies  developed  and 
refined  in  the  SORAT  Program  have  been  adapted 
in  recent  years  to  apply  to  other  warfare  areas, 
ship  types,  and  to  commercial  applications. 


Examples  are  the  Operational  Readiness 
Assessment  and  Training  System  (0RAT$)  for 
Surface  ASW  (FF  1052,  DD  963,  and  FFG  7 
Classes),  the  Deployable  Acoustic  Readiness 
Training  System  (DARTS)  for  P-3C  Patrol  Plane 
Squadrons,  (the  Battle  Group  Readiness 
Assessment  and  Training  System  for  AAW  (Anti-Air 
Warfare  Commander  Self  Training  and  Assessment 
of  Readiness  (STAR)  Program),  the  ship-level 
Anti-Air  Warfare  Readiness  Assessment  and 
Training  Program  (AAW  RATS),  operator 
performance  assessment  for  power  plant  (nuclear 
and  fossil)  control  room  simulators,  and  the 
Readiness  Assessment  and  Training  System  for 
Submarines  of  the  Royal  Australian  Navy 
(POORATS).  Several  other  adaptations  are 
cu1  ently  underway  or  in  the  advanced  proposal 
stage,  including  RATS  programs  in  tne  areas  of 
Electronic  Warfare,  Navigation  and  Piloting,  and 
military  and  civil  disaster  preparedness. 

Each  of  these  programs  has  b-vn  developed 
independently  and  each  has  presented  unique 
problems  to  the  program  and  material 
developers.  There  are,  however,  common  or 
"generic"  problems  and  difficulties,  normally 
encountered  in  every  project,  that  must  be 
addressed.  The  intent  of  this  paper  is, 
therefore,  to  describe  some  of  those  problems, 
sugqest  possible  resolutions  to  these  problems, 
and  to  then  list  a  number  of  thumbrules  which 
migh*  be  of  assistance  to  others  in  the 
development  of  onboard  training  and/or 
assessment  programs. 

onboard  training 

What  Is  On board  Training? 

Befc-e  proceeding  farther,  it  is  necessary 
to  define  "onboard  training"  as  the  term  will  be 
used  in  this  paper.  Much  discussion  has 
centered  around  onboard  training  in  recent  years 
and  there  is  confusion  about  what  onboard 
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training  encompasses.  In  a  general  sense, 
onboard  training  can  be  thought  of  as  any  type 
of  training  conducted  on  or  for  a  unit  outside 
of  the  shore-based  training  environment.  This 
definition  would  therefore  inc^de  such  training 
as  watchstation  training,  rate  training, 
lectures,  seminars,  demonstrations,  drills,  and 
on-the-job  training  (OJT),  both  formal  and 
informal.  When  onboard  training  is  viewed  by 
any  particular  individual,  however,  the 
definition  evoked  usually  applies  only  to  a  very 
specific  area,  usually  relating  to  that 
individual's  somewhat  parochial  interests.  For 
example,  to  a  Division  Officer,  the  quarterly 
schedule  of  lectures  for  the  division  may  be 
perceived  as  the  extent  of  onboard  training.  To 
an  ILS  manager  at  NAVSEA,  onboard  training  may 
relate  only  to  the  training  of  maintenance 
personnel  to  repair  his  particular  piece  of 
equipment.  To  tne  educational  specialist  at 
CNET,  the  term  might  only  apply  to  the  area  of 
OJT.  To  the  type  or  force  commander,  hopefully 
the  term  would  apply  to  any  type  of  training 
conducted  by  the  ship  to  establish  and  maintain 
its  proficiency  in  operations,  maintenance,  and 
overall  readiness.  The  very  general  term 
"onboard  training"  is  used  frequently  to 
describe  a  very  specific  aspect  of  this 
training. 

This  paper  will  also  apply  the  term  to  a 
very  narrowly  defined  area.  Onboard  training  is 
therefore  defined  as  "individual  or  team 
watchstation  training  designed  to  provide 
operational  fami 1 iarization  or  proficiency 
maintenance  in  those  skills  required  for  proper 
system  employment,  using  a.tual  equipment,  where 
possible,  and  simulating  the  at-sea  environment 
real istical ly."  It  should  be  noted  that  this 
rather  narrow  definition  excludes  equipment 
maintenance  training  and  other  types  of  training 
such  as  lectures  and  seminars  (although  it  could 
be  readily  expanded  to  encompass  these  also). 

It  does,  however,  closely  approximate  formal  OJT 
and  can  be  thought  of  as  a  supplement  to  this 
type  of  training. 

One  might  wonder  why  such  a  narrow  and 
restrictive  definition  of  "onboard  training"  is 
used.  Not  using  such  a  sharply  defined  term 
leads  to  the  most  formidable  problem  in 
development  of  onboard  train^nq  proqrams--narrel y 
that  a  program  cannot  be  all  things  to  ail 
people  at  all  times.  In  proposing  an  onboard 
training  program  to  be  administered  to  a  certain 
group  on  board  a  ship  (the  sonar  division,  as  an 
example),  the  developer  is  faced  with  a  diverse 
array  of  individuals  with  widely  varying  formal 
training,  at-sea  experience,  attitudes,  rates, 
arid  proficiency.  In  addition,  there  are  a 
variety  of  skills  an  operator  should  possess  in 
order  to  properly  accomplish  his  assigned 
watchstation  tasks.  The  developer  must 
structure  his  program  $0  *s  tn  addrps<.  th*se 
diversities  between  individuals,  as  well  as  the 
different  skill  areas. 

To  present  an  example,  take  the  sonar 
division  aboard  a  U.S.  nuclear  attack 
submarine.  Tne  division  as  a  whole  must  be 
proficient  at  detection,  classif ication, 
tracking,  and  specialized  operating 
procedures.  Further  breakdown  includes  both 


passive  and  active  operational  skills. 

Additional  subsets  of  skills  would  include 
torpedo  detection  and  tracking,  and  reporting 
procedures.  The  division  is  split  into 
watchsections,  so  proficiency  is  required  for 
several  groups,  as  well  as  for  the  Battle 
Stations  organization.  Skills  must  be  developed 
at  an  individual  operator  level,  as  we‘1  as  for 
the  team.  Team  skills  can  be  assembled  from 
individual  skills  only  with  effective 
coordination  and  supervision.  Therefore, 
supervisory  training  must  also  be  addressed. 

The  entry  level  of  personnel  into  this  division 
is  another  consideration.  The  division  chief 
and  experienced  supervisors  do  not  require  the 
same  training  that  a  seaman  or  junior  petty 
officer  should  obtain.  At  any  particular  level 
of  experience,  materials  should  offer  a  variety 
of  situations  or  levels  of  difficulty  so  that 
experience  can  be  acquired  on  increasingly  more 
difficuH  tasks.  The  list  could  go  on  and  on 
with  further  breakdown  into  desired  or  necessary 
areas  of  training. 

All  of  this  suggests  an  extremely  large, 
complex,  and  expensive  series  of  training 
programs,  if  all  of  these  considerations  are  to 
be  addressed.  Obviously,  such  a  series  of 
programs  would  be  highly  desirable,  but  the  cost 
of  developing  such  a  program  (for  only  one 
division)  would  be  staggering.  The  developer  is 
therefore  left  to  determine  which  areas  should 
be  addressed,  in  what  depth,  at  what  level  of 
difficulty,  and  in  what  manner  (medium  and 
format),  all  within  what  is  normally  a  very 
limited  budget.  He  must  then  wrestle  with  the 
problems  of  how  to  introduce  the  materials,  how 
to  promote  their  use,  and  how  to  maintain  the 
currency  of  the  program.  Obviously,  there  will 
be  a  wide  range  of  opinions  on  which,  what,  and 
how,  both  internal  to  the  developing 
organization,  and  external ly  within  the  client's 
organization. 

Some  of  these  specific  problems  which 
develop,  and  which  must  be  addressed,  will  be 
elaborated  on  in  the  remainder  of  this  paper. 
They  are  broken  down  into  five  major  areas:  1) 
developmental  problems,  2)  implementation 
problems,  3)  usaqe  problems,  4)  program 
maintenance  problems,  and  5)  Navy  policy  toward 
onboard  training. 

PROBLEM  AREAS 
Developmental  Problems 

As  alluded  to  before,  specification  of  a 
program,  currculum,  or  a  framework  for  future 
development  for  onboard  training  is  not  an  easy 
ta<-k.  T^e  traditional  Instructional  Systems 
Development  (ISD)  model  lends  itself  more  to  the 
development  of  a  formal  school -house  course  of 
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administered  traininq  program.  Since  the  combat 
systems  to  be  addressed  are  normally  already 
installed  in  operational  units,  it  is  desirable 
to  qet  training  materials  in  place  in  a  short 
period  of  tine.  Unless  much  of  the  front-end 
analysis  has  been  conducted  during  system  design 
and  development,  ihe  developer  is  faced  with  the 
lengthy  and  t  i.ne-consuming  process  of  task 
analysis  which  must  be  accomplished  before 
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proceeding  with  training  program  development. 
Within  time  and  funding  constraints  imposed  by 
the  client,  the  developer  must  also  decide  which 
areas  should  be  addressed  and  to  what  depth. 
Obviously,  these  conflicting  requirements  can 
cause  many  headaches  and  much  consternation  on 
the  part  of  developer  and  client  alike. 

Our  approach  to  this  problem  has  been  to 
let  the  client  decide.  Since  the  type  or  force 
commander  is  concerned  with  the  ability  of  his 
ships  to  go  to  wai  today  (and  not  three  years 
from  now)  thr  emphasis  has  usually  been  placed 
on  rapid  delivery  of  training  materials,  using 
existing  tactical  doctrine  and  operating 
procedures  as  the  documentation  of  correct 
operator  procedure  or  response  or  as  a 
performance  standard.  Using  this  approach, 
material  delivery  can  occur  early  in  program 
development  while,  at  the  same  time,  details  of 
the  long-term  program  can  be  filled  in  as  it 
progresses  Although  not  an  ideal  situation  for 
development  of  a  comprehensive  program,  it  noes 
provide  the  operational  commander  with  materials 
addressing  current  needs,  while  at  the  same  time 
developing  a  framework  for  addressing  future 
systems  and  requirements. 

Even  when  the  framework  has  been  developed, 
front-end  analysis  completed,  objectives 
specified,  and  media  determined,  the  training 
specialist  who  will  develop  the  materials  is 
still  faced  with  a  multitude  of  problems. 

Within  the  sphere  of  operational  combat  system 
training,  these  can  be  placed  into  two  general 
cateqories--l ack  of  documentation  and  equipment 
1  imitations. 

Lack  of  Documentation.  Perhaps  the  most 
frustrating  situation" for  a  training  raterial 
developer  is  to  not  have  adequate  technical  and 
tactical  documentation  to  support  training 
material  development.  This  situation  has  arisen 
for  virtually  every  new  combat  system  delivered 
to  the  Navy  in  the  past  several  years.  Besides 
preliminary  operating  guidelines  (based  on  how  a 
system  should  work)  and  technical  manuals  (often 
with  many  "To  Be  Supplied"  pages),  little 
written  documentation  is  available  to  assist  the 
develoD<r  in  creating  the  materials.  Our 
approach  to  this  problem  has  been  to  employ 
subject  matter  experts,  trained  in  the  field  of 
interest,  to  generate  the  training  materials. 
These  material  developers  work  closely,  and 
interact  frequently,  with  users  and  technical 
experts  to  verify  accuracy  of  the  developed 
materials.  As  Fleet  experience  in  operating  the 
system  is  obtained  and  as  tactical  procedures 
are  refined,  identified  improvements  can  be 
readily  incorporated  into  the  existing 
materials. 

Conflicting  Tactical  Doctrine.  A  problem 
frequently  encountered  is  that  o7  conflicting 
documentation  for  a  system.  This  is  sometimes 
caused  by  the  delay  in  delivering  documentation 
for  a  system,  or  by  making  a  change  in  one 
publication  and  not  another.  Similar 
discrepancies  often  develop  because  of  official 
or  unofficial  differences  between  Pacific  and 
Atlantic  Fleet  operating  environments  a  d 
tactics.  There  arc  two  approaches  which  can  be 
used  to  alleviate  this  problem.  One  is  to 


create  materials  that  are  non-specific  in  the 
areas  of  conflict.  The  other  is  to  develop  two 
versions  of  the  same  module,  each  version 
reflecting  the  applicable  guidance.  Although 
the  latter  is  the  better  alternative,  it  creates 
administrative  problems  in  dissemination  and 
control  of  materials.  Therefore,  the  "generic" 
training  material  approach  is  generally  used 
with  space  allowed  foi  insertion  of  locally 
determined  amplifying  material. 

Imbedded  Training  Capability  Problems.  One 
of  the  cornerstones  of  111  of  our  programs  has 
been  reliance  on  imbedded  training  modes  and 
inherent  stimulation  capabilities  of  the 
systems.  Based  on  the  premise  that  the  most 
effective  operational  training  should  be 
conducted  on  an  operator's  own  equipment  in 
realistic  surroundings,  built-in  training  modes 
are  used  whenever  possible.  Although  many  of 
these  training  modes  provide  excellent  high 
fidelity  stimulation,  virtually  all  are  somewhat 
limited  and  their  capability  must  be 
supplemented  to  provide  additional  realism  and 
effective  training.  The  first  characteristic 
weakness  observed  in  virtually  all  imbedded 
systems  is  that  they  have  been  designed  solely 
for  stimulation  of  that  particular  piece  of 
equipment  and  no  others.  For  example,  the 
built-in  training  mode  of  the  MK  117  Fire 
Control  System  provides  a  training  capability 
for  Weapon  Control  Console  (w'CC)  operators, 
especially  in  the  ai  ea  of  Tarqet  Motion 
Analysis.  This  mode  is  adequate  for  training 
individual  WCC  operators  but  has  severe 
shortcomings  if  Attack  Team  training  is 
desired.  It  is  not  designed  to  drive  plotters 
or  bearing  repeaters  and  so  the  Plotting  Team  is 
effectively  excluded  from  this  team  training 
unless  some  artifice  is  used  to  include 
realistic  sensor  inputs.  Likewise,  the  'mbedded 
training  mode  of  the  AN/BOO-5  Sonar  System 
provides  an  unsatisfactory  presentation  and  a 
small  variety  of  contacts  and  must  somehow  be 
si  pplemented  or  substituted  for.  In  both  of 
these  situations,  solutions  to  the  problems  have 
been  developed  which  have  not  significant ly 
detracted  from  realism.  Concurrent  operation  of 
the  sonar  and  fire  control  systems  has  presented 
considerable  problems,  however.  Due  to  the  many 
artificialities  required  by  simultaneous 
operation  for  traininq,  the  number  ot  personnel 
required  to  administer  a  combined  team  traininq 
exercise  approaches  the  number  of  exercise 
participants  or  trainees  ana  is  thus  normally 
impractical  to  conduct.  Designers  of  future 
systems  should  consider  the  need  for  complete 
combat  system  team  traininq  capability 
(including  operation  in  back-up  or  casualty 
modes)  in  desiqninq  imbedded  traininq  functions 
for  these  systems. 

Another  deficiency  frequently  observed  with 
imbedded  traininq  functions  is  the  shortcomings, 
or  total  lack,  of  operational  checks  ana 
preventive  maintenance  cover inq  these  modes  and 
equipment.  This  is  particularly  true  of  some  of 
the  older  analog,  mechanical  systems.  One  prime 
example  is  the  Own  Sh>p  Motion  Simulator  (0SVOS) 
used  to  provide  own  ship  course  and  speed  inputs 
to  fire  control  and  plottinq  equipment  aboard  FF 
105?  Class  ships.  On  virtually  every  ship  on 
which  the  ORATS  program  was  demonstrated  the 
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OSMOS  either  did  not  operate  at  all,  or  provided 
only  partial  or  improper  input  to  the 
equipment.  In  all  cases,  this  deficiency  was 
unknown  to  ship's  personnel  because  there  was  no 
periodic  preventive  maintenance  to  check  out  the 
system  and  testing  of  the  unit  was  not  adequate 
during  new  construction  or  overhaul.  To  solve 
this  *ype  of  problem,  it  has  often  bpen 
necessary  to  develop  special  testing  and 
alignment  procedures  as  adjuncts  to  training 
materials  for  this  equipment.  These  are 
conducted  periodically  by  thp  ship  to  verify 
proper  operation  and  identify  potential  faults 
or  problems. 

The  long  term  solution  to  all  of  these 
problems  is,  of  course,  to  build  a  comprehensive 
training  stimulation  and  management  capability 
into  each  new  combat  system.  This  process 
should  ideally  commence  on  Day  1  of  system 
specification,  and  be  carried  through  full  scale 
development  with  the  same  priority  and  emphasis 
as  with  any  other  part  of  the  combat  system. 
Recent  events  in  procurement  of  systems  such  as 
the  AN/SQQ-89  and  the  SUBACS  have  shown 
encouraging  signs  that  the  Navy  is  moving  in 
this  direction. 


System  Upgrading  and  Improv ement 
Problems.  A  constant  thorn  in  the  side  of 
training  material  developers  (for  which  there  is 
currently  no  good  solution)  is  the  seemingly 
endless  series  of  changes  and  updates  to  current 
combat  systems.  Take  for  example  the  AN/BQQ-5 
Sonar  System.  There  are  three  versions  of  the 
system  (Baseline,  A,  and  B)  installed  on 
submarines  today  with  another  scheduled  for 
introduction  in  the  next  few  years  (C).  For  a 
developer  to  produce  materials  for  each  is,  in 
itself,  a  formidable  task.  Add  to  this  a  series 
of  updates  (FY-79  update,  FY-81  update,  etc.), 
each  of  which  modifies  operation  of  the 
system.  Now  the  developer  must  choose  between 
creating  training  materials  so  general  that  they 
can  cover  all  systems  or  creating  several 
different  versions  of  each  module  detailed  to 
'ddress  all  configurations.  W'th  the  large 
number  of  systems  in  use,  the  administrative 
burden  of  keeping  track  of  who  has  what  is 
staggering. 

A  variation  of  this  same  problem  recently 
occurred  when  a  software  change  to  a  combat 
system  rendered  over  50*  of  the  training 
materials  in  the  Fleet  addressing  that  system 
obsolete.  At  present  the  only  solution  to  this 
problem  is  to  withdraw  the  obsolete  materials 
from  circulation  and  gradually  replace  them  as 
time  and  funding  permit. 
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When  the  training  materials  have  been 
produced,  the  overall  job  is  only  partially 
complete.  The  materials  must  be  introduced, 
demonstrated,  and  integrated  into  a  ship's 
existing  training  program.  A  well  conceived, 
timely,  and  efficient  implementation  plan  is 
essential  to  the  long-term  success  oT  any 
training  program  or  training  materials  which 
will  be  used  by  operating  forces  on  board  ship. 


A  well-conceived  plan  is  one  which  will 
provide  exposure  of  the  program  to  the  largest 
number  of  units,  provide  a  comprehensive  view  of 
the  overall  program  and  specific  examples  of 
materials,  place  the  least  possible 
administrative  burden  on  the  ship,  and  convince 
personnel  on  the  snip  that  the  program  can,  in 
fact,  help  them  upgrade  and  maintain  their 
operational  readiness. 

In  this  area,  advanced  planning  is  the  key 
element  to  avoid  problems.  This  planning  should 
be  done  with  the  cognizant  squadron  or  force 
representat~ ves  in  order  to  facilitate 
scheduling  of  individual  units.  A  detailed 
agenda  of  each  proposed  ship  visit  should  be 
prepared  and  submitted  to  the  squadron  for 
review  and  comment.  Examples  of  items  which 
should  be  covered  in  the  implementation  visit 
include  briefings  on  program  purpose, 
objectives,  mechanics,  and  methods  structured 
toward  a  particular  audience  (e.g..  Wardroom, 
Sonar  Division,  Section  Tracking  Parties). 

Then,  representati ve  examples  of  the  training 
materials  contained  in  the  program  should  be 
demonstrated.  As  an  example,  during  SORAT 
Program  implementation  visits,  individual  as 
well  as  team  sonar  and  attack  team  exercises  are 
conducted  on  each  ship.  Finally,  time  should  be 
set  aside  to  assist  division  officers  and 
leading  petty  officers  by  suggesting  methods  for 
integration  of  the  trair  g  materials  into 
existing  training  plan,  and  qualification 
processes. 

A  key  factor  of  this  advanced  planning  is 
selection  of  personnel  to  conduct  the 
implementation.  The  single  most  important 
attribute  of  these  personnel  is  credibility, 
followed  closely  by  the  ability  to  communicate 
with  all  echelons  of  the  Navy  structure,  from 
junior  enlisted  to  flag  officer.  The 
implementer 's  cred;bility  includes  knowing  the 
program,  knowing  t!  e  audience,  ;;nd  being  able  to 
relate  the  conditions  and  environment  of  the 
ship  to  his  own  experience.  The  ability  to 
communicate  effectively  is,  in  fact,  a  major 
part  of  the  individual's  credibility,  and 
implies  the  ability  to  discuss  the  specific 
aspects  of  the  proqram  in  layman's  terms,  not 
those  of  academia  or  the  educational  community. 

The  second  element  of  a  successful 
implementation  is  timing.  Cominq  aboard  a  ship 
two  days  before  a  major  inspection  or  durinq 
loadout  for  a  deployment  can  totally  destroy  anv 
positive  attitude  toward  a  proqram.  There  is  no 
single  best  time  to  plan  for  implementation,  but 
ideally  the  period  shortly  after  a  major 
overhaul  is  thp  best.  Personnel  a^e  normally 
anxious  to  go  to  sea,  and  desire  to  hone  their 
previously  established  operating  skills  which 
have  been  dulled  by  the  long  in  port  period. 
Obviously,  if  implementation  is  conducted  on  a 
squadron  basis,  not  all  ships  will  be  at  the 
same  point  in  the  overhaul  cycle.  Rut  in 
establishing  a  lonq  term,  individualized 
program,  this  post-overhaul  perioo  is  ideal  for 
introducing  a  proqram. 

The  third  element  in  successful 
implementation  is  efficiency.  This  depends  to  a 
larqe  extent  on  the  advanced  planning  which  has 
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been  done.  The  flexibility  of  implementation 
team  personnel  is  also  a  major  factor.  The 
ability  to  respond  professionally  and 
knowledgably  to  a  particular  problem  reflects 
well  on  the  program  as  a  whole.  For  example, 
suppose  a  particular  demonstration  scheduled  in 
the  agenda  cannot  be  accomplished  because  the 
equipment  is  down  for  maintenance.  The  team 
should  be  able  to  suggest  and  conduct  alternate 
demonstrations  not  requiring  use  of  that  piece 
of  equipment.  The  crew's  time  is  valuable  and 
any  real  or  perceived  waste  of  that  time  will 
detract  from  acceptance  of  the  program  or 
materials. 

The  final  element  of  success  is  to  get 
squadron  personnel  to  assist,  or  at  lea't  be 
present,  for  each  implementation  visit.  In 
addition  to  adding  an  air  of  squadron  interest 
and  participation  to  the  implementation,  it  also 
serves  to  indoctrinate  and  train  on-site  program 
facilitators  who  will  be  able  to  provide 
implementation  visits  on  their  own  to  units  not 
present  when  the  original  implementation  is 
conducted. 

Usage  Problems 

It  is  not  uncommon  to  hear  from  ■  train ir.q 
material  or  program  developer  the  plaintive  cry, 
"I  pour  my  heart  and  soul  into  developing  the 
best  possible  training  materials,  show  the  F'?et 
how  to  use  them,  and  convince  them  that  they  can 
increase  their  readiness  by  using  the  materials, 
and  then  the  Fleet  doesn't  us?  them!  Why?"  The 
answer  is  simple-- lack  of  time.  Anyone  who  has 
served  on  a  fast  attack  submarine  knows  that  the 
acronym,  SSN,  does  frequently  stand  for 
"Saturdays,  Sundays,  and  Nights."  The  same 
holds  true  for  virtually  every  other  platform  in 
the  Navy.  With  heavy  preventive  and  corrective 
maintenance  schedules,  shortaqes  in  manninq, 
hectic  operating  schedules,  and  a  host  of  other 
requirements,  both  inport  and  at  sea,  the  time 
available  for  onboard  traininq  is  extremely 
limited.  On  a  list  of  priorities  for  an  upkeep, 
traininq  would  probably  come  last,  and  would  be 
the  first  item  to  slip  off  the  list  as  the  time 
to  get  underway  approaches.  Therefore,  how  can 
onboard  training  program  usage  be  initiated  and 
maintained? 

Part  of  the  responsibility  for  this  iies, 
of  course,  directly  on  the  shoulders  ot  the 
Navy.  Emphasis  on  the  importance  of  traininq 
and  perhaps  even  a  mandate  for  its  use  will 
provide  part  of  the  answer.  However,  it  is  also 
the  responsibility  of  the  material  and  proqram 
developer  to  assist  in  this  by  producing  a 
product  which  is  responsive  to  shipboird 
application  and  use.  In  our  experience,  several 
factors  most  directly  affect  shipboard  attitudes 
and  usage.  The  most  significant  of  these  are 
realism,  administrative  burden,  and  user 
feedback . 

A  trainee  will  often  make  a  comment  such 
as,  "This  contact  doesn't  sound  (or  look)  like 
the  real  ones  on  my  system."  Because  of  this 
discrepancy  between  what  is  contained  in  the 
traininq  material  and  what  the  trainee  perceives 
as  the  real  w or5d,  he  will  often  "turn  otT"~  to 
the  material,  fegaruM-ss  of  the  we*  i  red 


objective  of  the  material  or  of  its  inherent 
importance  or  training  value.  Therefore,  care 
must  be  taken  in  constructing  any  operational 
training  material  to  ensure  that  it  accurately 
reflects  the  environment,  contacts,  procedures, 
tactics,  and  situations  encountered  in  actual 
operation.  This  is  normally  best  assured  by  a 
rigid  and  comprehensive  validation  process  which 
consists  of  technical  (does  this  exercise 
proceed  as  it  was  desiqned  to?)  and  content 
(does  it  meet  its  stated  learning  objectives?) 
validations  in  conjunction  with  Navy  subject 
matter  experts. 

The  second  important  factor  is  the 
administrative  burden  placed  on  the  ship  by 
implementing  or  integrating  a  formal  onboard 
training  program.  It  is  our  firm  belief  that 
any  program  imposed  as  an  addi t i onal  training 
requirement  is  instantly  doomed  to  failure. 
Rather,  the  program  should  be  developed  to: 
replace  some  aspect  of  presently  required 
training  (conducting  an  onboard  attack  team 
exercise  in  lieu  of  one  attack  trainer  session); 
enhance  traininq  within  an  existing  framework  (a 
sonar  team  classification  exercise  in  lieu  of  a 
divisional  lecture  on  classification);  support 
the  watchstation  qualification  process 
(materials  designed  to  prepare  a  trainee  for 
performance  of  a  practical  factor);  support  ship 
qualification  or  advancement  in  rate;  and 
support  at-sea  watchsection  traininq  without 
interrupting  normal  routine. 

User  feedback  is  another  important 
aspect.  If  personnel  in  the  Fleet  are  solicited 
for  input  and  comments  concerning  the  program  or 
its  materials,  a  much  greater  appreciation  of 
its  existence  can  be  developed.  In  fact, 
incorporating  Fleet  comments  and  recommendations 
into  the  program  can  result  in  a  "pride  of 
authorship"  situation  which  also  promotes  wider 
usage.  Perhaps  many  would  brush  aside  this 
factor  as  simply  "stroking  the  egos"  of  the 
users,  but  in  the  long  run  it  provides  a  program 
responsive  to  current  Fleet  needs,  in  a  form  and 
package  suitable  for  (and  conducive  to,  use  on 
board.  We  strongly  recommended  that  any  program 
designed  for  use  on  board  include  input  from  the 
user  and  provide  a  viable  feedback  process  to 
make  this  happen. 

Na vy  Policy  Toward  Onboard  Training 

Despite  a  stated  stronq  commitment  to 
onboard  training  by  the  Deputy  Chiefs  of  Naval 
Operations  (Submarine  and  Surface  Warfare),  as 
yet  there  exists  no  Navy-wide  or  ship-type-wide 
policy  addressinq  it.  Several  factors  are 
responsible  for  this.  First  is  the  relative 
newness  of  the  concept  itself --that  is,  training 
of  at-sea  forces  usiig  ojtside  assets.  Since 
the  establishment  of  the  U.S.  Navy,  the  concept 
that  the  Cornandinq  Officer  is  ultimately 
responsible  for  the  training  and  readiness  of 
his  ship  has  been  a  guidinq  principle.  This  is 
no  less  true  today  than  it  was  two  hundred  years 
ago.  But  now,  the  systems  at  the  disposal  of 
the  CO  are  much  more  complex,  the  enemy  is 
equipped  with  comparable  system,,  the 
administrative  burdens  are  increased 
dramatically,  requiring  the  CO  tc  have  outside 
assistance.  Thic  assist anc*.  in  vhe  form  of 
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onboard  training  materials  and  programs,  must  be 
coordinated  and  well  planned,  and  policy 
concerning  this  should  be  detc-rmined  at  the 
highest  levels  of  the  Navy.  This  plan  must 
consider  integration  of  existing  formal  school 
training,  shore-based  operational  training, 
refresher  training,  and  present  normal 
operational  training  exercises.  Several  type 
commanders  have  made  admirable  headway  in 
developing  such  coordinated  programs.  Most 
notable  are  COMSUBPAC's  PORT  and  STBD  program 
and  COMSURFPAC's  Combat  System  Training 
Architecture  program  which  define  cyclical 
training  requirements  for  ships  between 
overhauls.  However,  a  Navy-wide  program  or 
policy  should  be  estabished  to  ensure  conformity 
and  standardization. 

A  second  problem  concerning  onboard 
training  policy  is  that  there  is  no  one  single 
office  within  the  Navy  wnich  is  tasked  to 
control  ur  coordinate  the  area  of  onboard 
training.  Currently  there  is  much  confusion 
about  who  has  cognizance  over  what  aspect  of 
onboard  training  (and  trainers  for  that 
matter).  Warfare  commanders,  type  commanders, 
NAVSEA,  NAVAIR,  NAVELZX,  NTEC,  CNET,  and  other 
Navy  training  commands--al 1  of  these  have 
somewhat  different  perceptions  uf  what  onboard 
training  is  and  should  be,  and  who  is 
responsible  for  each  aspect.  A  hiqh  level 
determination  of  which  organizations  should 
exercise  cognizance  over  the  various  aspects  of 
onboard  training  should  be  made  as  part  of  an 
overall  policy  statement. 

The  final  policy-related  problem  to  be 
discussed,  one  which  is  dear  to  the  heart  of 
every  contractor  engaged  in  onboard  training 
program  support,  is  that  of  funding.  Funding 
wise,  onboard  training  programs  are  true 
orphans.  Presently,  most  funds  earmarked  for 
onboard  training  are  O&M,  N  which  are  very 
short-lived  and  cut  into  the  operating  budgets 
of  the  type  commanders.  In  this  era  of  major 
shipbuilding  and  weaoons  procurement,  stringent 
oversight  and  budgetary  restraint,  the  first 
.unds  *o  be  cut  will  be  from  O&M,  N  monies. 
Thereto *e,  type  commanders  are  faced  with  the 
unenviable  choice  of  keeping  their  ships  at  sea 
by  funding  reeded  maintenance,  or  providing 
additional  training  to  upgrade  and  maintain 
personnel  readiness. 

The  perishability  of  this  fundinq  (with  its 
one-year  lifetime)  presents  problems  to  the  Navy 
sponsor,  as  well  as  the  contractor,  in  settinq 
up  and  maintaining  a  long-term,  systematic 
program.  As  part  of  a  desired  Navy-wide  policy, 
funding  for  development  of  lonq-term  onboard 
traininq  programs  should  be  addressed,  just  as 
life  cycle  supply  support  is  addressed  in 
procurement  of  a  new  combat  system. 

SUMMARY 

Onboard  traininq,  particularly  onboard 
traininq  which  is  supported  lonq-term  and  is 
part  of  an  overall  Navy-wide  approach  to 
maintaining  a  high  state  of  operational 
readiness,  is  a  concept  whose  time  has  come. 
Although  development  of  onboard  training 
programs  has  been,  and  will  continue  to  be,  a 


challenging  anu  often  frustrating  task,  the 
ultimate  rewards  and  satisfaction  can  be  great-- 
both  for  the  Navy  and  the  training  material 
developer.  It  has  been  one  intent  of  this  paper 
to  show  that  all  of  these  past,  present,  and 
future  programs  are  evolutionary,  solely  because 
of  the  nature  of  the  beast.  And  they  will 
continue  to  change  and  improve  as  more 
experience  is  gained  in  this  area. 

In  conclusion,  a  list  of  thumbrules  for  the 
program  or  material  developer  is  provided  which, 
hopefully,  can  be  of  use  to  others  embarking  on 
onboard  training  program  development. 

1.  Don't  underestimate  the  abilities  of 
the  audience.  Today's  sailor  or  soldier  is 
intelligent,  alert,  and  highly  trainable. 

2.  Don't  anticipate  the  requirements  of 
the  user.  Get  down  to  the  waterfront  and 
find  out  for  yourself. 

3.  Enlist  the  participation  and  input  of 
operating  Navy  personnel  whenever  possible. 

4.  Don't  bite  off  more  than  you  can 

chew.  Don't  promise  a  panacea  when  you  are 
only  delivering  a  Band-Aid. 

5.  Likewise,  don't  deliver  a  body  cast 
when  a  Band-Aid  wi 11  do. 

6.  Remember  when  you  go  aboard  a  ship, 
that  this  is  the  sailor's  home  and  he 
expects  you  to  treat  it  as  such. 

7.  Don't  be  disappointed  when  someone 
doesn't  think  your  materials  are  as  good  as 
you  do.  Someone,  somewhere,  can  always  do 
it  better. 

8.  Know  the  environment  that  your 
materials  will  be  used  in  (on  board  the 
ship)  and  develop  them  so  that  they  can  be 
jsed  there. 
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ABSTRACT 


The  ship  bridge/shiphandling  simulator,  with  a  large  visual  scene,  is  a  relatively  new 
development  for  the  training  of  deck  officers.  The  U.S.  Coast  Guard  and  U.S.  Maritime 
Administration  have  jointly  sponsored  the  Training  and  Licensing  Project  to  investigate 
the  role  of  the  ship  bridge/shiphandl  ing  simulator  in  the  training  and  licensing  of  deck 
officers.  A  major  product  stemming  from  this  multi-year  research  is  a  set  of  guidelines 
ror  the  development  of  deck  officer  training  systems.  The  guidelines  address  the 
simulator/trairing  device,  the  training  program,  and  the  instructor.  The  guidelines  are 
intended  for  use  by  operational  organizations  to  assist  in  determining  the  adequacy  of 
available  simulator-based  training  programs  to  fit  their  specific  needs,  and  to  assist  in 
specifying  training  systems  for  procurement.  The  guidelines  have  a  generic  structure  to 
address  all  levels  of  deck  officer  training.  The  specific  content  of  the  guidelines  was 
initially  developed  to  address  masters-level  training.  The  initial  direct  application  of 
the  guidelines,  however,  was  for  the  design  of  a  simulator  for  Maritime  Academy  cadet 
training.  Each  of  these  are  briefly  discussed. 


INTRODUCTION 

National  and  International  concern  regarding 
the  safety  or  merchant  vessels,  crews,  and 
cargoes  has  increased  substantially  in  recent 
years.  Th<s  increasing  concern  is  occurring 
with  the  public,  regulatory  agencies,  shipping 
companies  and  the  general  body  of  mariners 
themselves.  As  the  productivity  of  shipping 
has  increased  over  the  years,  so  have  the  risks 
associated  with  shipping,  the  difficulty  of 
operations  and  the  requisite  level  of  mariner 
skill  required,  and  also  the  potential  damage 
and  cost  of  accidents.  Whereas  the  maximum 
size  of  ships  afloat  50  years  ago  was  6000  dead 
weight  tons  (dwt),  the  larger  ships  float  today 
exceed  500,000  dwt.(2U  The  allowable  margin 
of  error  in  maneuvering  a  vessel  has  generally 
decreased  as  the  vessel  size  increased;  the 
amount  of  hazardous  cargo  carried  hjs  generally 
increased,  particularly  in  the  larger  vessels 
(e.g.,  crude  oil,  liquified  natural  gas).  As 
might  be  expected,  the  annual  tonnage  loss  has 
also  steadily  increased.  Whereas  in  1962  about 
125  ships  were  lost,  representing  about  525,000 
gross  tons,  in  1977  slightly  more  than  200 
ships  were  lost  representing  more  than 
1,200,000  gross  tons.v1^)  These  increases 
represent  substantial  financial  loss,  wd  often 
catastrophic  loss  of  life  and  damage  to  the 
environment.  Several  investigations  into  the 
cause  of  maritime  accidents  have  indicated  that 
as  much  as  80  percent  of  the  contributory 
causes  mav  ..,Araceab1e  t0  *,uman 

performance. '*), ( 1 3) , (22)  in  addition, 

several  other  investigations  have  sought  to 
delineate  the  specific  aspects  of  human 
oerformance,  as  well  as  other  factors,  that  are 
prime  ,  contributors  to  maritime 

accidents.  ,  (16)  As  a  result  of  these 
factors,  considerable  emphasis  has  been  placed 


by  all  segments  of  the  maritime  industry  on  the 
improvement  of  mariner  proficiency  including 
better  human  factors  design  of  the  man-ship 
interface  and  improved  training  programs. 

The  ship  bridge/shiphandl ing  simulator  is  a 
tool  that  has  recently  become  available  to 
assist  in  the  improvement  of  deck  officer 
training.  The  first  shiphandling  simulator, 
the  Marine  Research  and  Training  Center  of  Port 
Revel  near  Grenoble,  France,  was  built  in 
1967.  It  consists  of  an  eight  acre  lake  with 
1:25  scale  model  ships  in  which  ‘.wo  trainees 
sit,  and  maneuver  around  the  lake.  The  first 
electronic  ship  bridge/shiphandling  simulator 
became  operational  in  19o8  at  the  Institute  TNG 
for  Mechanical  Constructions  in  the 
Netherlands.  The  predominant  difficulty  in  the 
development  of  the  ship  bridge/shiphandling 
simulator  has  been,  and  stir  is,  the  visual 
scene  technology.  A  large  visual  scene,  of  up 
to  360  degrees  horizontal  field  of  view  is 
necessary  in  this  type  of  a  simulator.  A 
variety  of  approaches  have  been  used  to  achieve 
a  visual  scene,  with  virtually  all  simulators 
substantially  differing  in  the  technology  used 
until  the  present  time.  More  recently, 
computer  generated  imagery  has  emeryeu  (i.e., 
within  the  past  5  years)  as  a  stable  technology 
for  this  type  of  visual  scene  simulation. 
Several  simulators  are  currently  operational 
with  computer  generated  imagery,  while  several 
others  are  in  the  planning  and  construction 
stages.  Hence,  the  recent  technological 
advances  in  simulation  have  resulted  in  a 
growing  use  and  acceptance  of  the  snip 
bridge/sh'phandl ing  simulator  as  a  viable  means 
for  improving  the  proficiency  of  deck 
officers.  The  growing  acceptance  of  this 
approach  to  training  is  underscored  by  recent 
national  and  international  actions: 
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•  The  U.S.  Port  and  Tanker  Safety  Act  of 
1978  calls  for  the  upgrading  of  deck 
officer  skills,  while  specifically 
calling  for  the  establishment  of 
standards  relating  to  "qualification 
for  licenses  by  use  of  simulators  for 
the  practice  or  demonstration  of 
marine  oriented  skills". (’9) 

•  The  U.S.  Coast  Guard  allowance  of 

successful  completion  of  a 
simulator-based  training  program  to 
satisfy  40  percent  of  the  pilot 
endorsement  requirements  for  the  Port 
of  Valdez,  Alaska. 

•  The  IMCO  Convention  on  the  Standards 

of  Training  and  Watchkeeping  (’4) 
recognizes  simulator-based  training  as 
a  means  of  acquiring  shiphandling 
skill  for  transitioning  to  a  vessel 
with  unfamiliar  handling 

characteristics. 

•  Europort  Pilots  have  been  undergoing 

simulator-based  training  periodically 
as  a  self-imposed  requirement  for 
practicing  and  improving  their 
shiphandling  skills. 

•  Shipping  companies  have  increasingly 

sought  the  use  of  simulator-based 
training  programs  for  their  deck 
officers.  Labor  unions,  likewise, 
have  recognized  the  potential  of 
simutator-based  training  and  have 
acquired  facilities  to  provide  sucn 
training,  as  well  as  encouraged  their 
members  to  utilize  the  facilities  of 
training  schools, 

Training  and  Licensing  Project 

As  a  result  of  the  growing  concern  about 
maritime  safety  and  productivity,  the  emergence 
of  acceptable  ship  bridge/shiphandling 
simulation  technology,  and  the  growing  interest 
in  simulator-based  training  for  deck  officers, 
the  U.S.  Coast  Guard  and  U.S.  Maritime 
Administration  embarked  on  a  joint 
Investigation  of  the  role  of  the  ship 
bridge/shiphandling  simulator  in  the  deck 
officers  training  and  licensing  process.  This 
multi  year  project  ( l . e . ,  Training  and 
Licensing  Project)  was  conducted  by  the 
Computer  Aided  Operations  Research  Facility 
(CAQRF)  of  the  National  Maritime  Research 
Center.  The  project  encompassed  several  phases 
which  addressed  simulator  design  and 
application  issues,  as  well  as  tne  range  of 
deck  officer  licensing  categories. 

The  problems  in  the  maritime  industry  relating 
to  the  design  and  use  of  simulator-based 
training  are  similar  to  those  encountered  in 
other  industries  which  draw  on  the  use  of 
complex  electronic  simulators.  They  are  (1) 
the  covt/effective  application  of  ship 
bridge/shiphandl ing  simulator  for  the  training 
of  deck  officers  (e.g.v  training  objectives 
that  can  be  effectively  met  by  such  training); 
(?)  the  cost/effective  design  character istics 
of  tne  simuistor/training  device  and  other 


parts  of  the  training  system  (i.e.,  training 
program  and  instructor),  and  (3)  the  effective 
design  characteristics  of  the  training  system 
with  regard  to  obtaining  some  type  of  deck 

officer  license  credit.  The  Training  and 

Licensing  Project  addressed  these  issues  in  a 
systematic  manner,  basing  its  investigation  on 
the  general  systems  approach  to  training  (e.g.. 
Instructional  Systems  Development  (ISD).tU 

The  initial  phase  of  the  Training  and  Licensing 
Project  (12)  (1}  developed  the  methodology  to 
be  used  throughout  the  duration  of  the 

investigation,  based  on  tailoring  the  systems 
approach  to  the  particular  needs  of  the 

Maritime  Industry;  (?)  developed  an  extensive 
information  base  addressing  deck  officer 
behavior  and  training  technology;  and  (3) 
identified  research  needs  to  adequately  address 
the  design  and  use  of  the  ship 
bridge/shiphandling  simulator  in  training. 

The  second  phase  of  the  program  conducted 
empirical  research  on  the  CAORF  ship 
bridge/shiphandl ing  simulator  to  investigate 
(1)  the  training  effectiveness  differences  of 
alternative  training  system  characteristic 
levels  of  fidelity  (e.g.,  color  versus  black 
and  white  visual  scene;  compressed  versus 
distributed  practice;  instructor  differences); 
and  (2)  the  training  effectiveness  of 
simulator-based  training  for  master's  level 
deck  officer  and  for  maritime  academy  cadets. 
A  variety  of  findings  were  forthcoming  from 
these  empirical  investigations. ( ^  T, ( 
These  investigations  address  the  application  of 
simulator-based  training,  and  the 
cost/effective  design  characteristics  tor 
achieving  an  effective  training  process. 

The  third  phase  of  the  program  was  devotee  to 
(1)  the  development  of  guidelines  for  master's 
level  deck  officer  training  systems,  and  (2) 
the  design  of  a  simulator/training  device  tor 
cadet  training  drawing  upo.i  the  extensive 
research  conducted  during  the  earlier  two 
phases.  Initially,  the  third  phase  was  going 
to  develop  criteria  for  evaluating  and 
approving  simulator-based  training  facilities 
as  meeting  some  minimum  standards  for  licensing 
purposes.  If  credit  were  to  be  gnen  towards 
some  licensing  requirement,  the  school  would 
have  .fo  be  certified  by  a  regulatory 
body.(^)  The  Training  and  Licensing  Project 
was  to  have  developed  training  system 
acceptance  criteria  by  which  this  evaluation 
could  be  accomplished.  bince  it  was  not 
feasible  to  make  simulator-based  training  a 
part  of  the  licensing  structure  at  this  tune, 
it  was  decided  to  develop  User  Guidelines  for 
t!ie  design  and  use  of  ship  bridge/shiphandling 
trainers  rather  than  criteria  for  their 
evaluation. 

The  Guidelines  for  Deck  Officer  Training 
Systems  w)  are  the  result  of  tne  earlier 
phases  of  research  on  the  Training  and 
Licensing  Project.  Ihey  are  intended  as  a 
user's  guide  to  the  design  and  use  of  the  ship 
br idge/shiphand1 ing  simulator  for  training  deck 
office-s.  in  nart  lcrlar,  the  guidelines 
provide  an  overall  structure  tor  the  design  of 
the  training  system  character ist ics  keyeJ  to 
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deck  officer  tasks.  The  guidelines  were 
initially  developed  for  the  training  of  senior 
mariners  (i.e.,  master's  level  deck  officers), 
hence  the  specific  details  of  guidance  provided 
address  the  master's  level.  The  guidelines 
identify  deck  officer  tasks,  indicate  those 
tasks  requiring  training  emphasis  via  analysis 
of  the  accident  literature,  identify  critical 
simulator  and  training  program  characteristics, 
provide  guidance  regarding  the  use  of  three 
distinct  levels  of  quality/fidelity  for  each 
characteristic  in  consonance  with  the 
objectives  aid  conditions  of  training  and 
recommend  a  set  of  minimum  level  simulator  and 
training  program  specifications  for  each  of 
five  major  training  modules.  The  training 
system  elements  addressed  include  the 
simulator/training  device,  training  assistance 
technology  to  be  included  as  part  of  the 
simulator/training  device  (e.g.,  feedback 
displays),  training  prugram  characteristics 
(e.g.,  instructors  guide),  and  instructor 
characteristics.  The  guide  is  intended  for  use 
by  operational  organizations  to  assist  ir 
determining  the  adequacy  of  available 
simulator-based  training  programs  to  fit  their 
specific  needs,  and  to  assist  in  specifying 
training  systems  for  procurement. 

TRAINING  SYSTEM  GUIDELINES 

The  guidelines  for  deck  officer  training 
systems  were  designed  around  a  general  purpose 
structure,  tailored  to  the  needs  and  resouces 
of  the  maritime  industry,  w'th  specific  content 
directed  at  the  master-level  mariner.  The 
structure  of  the  guideline:  were  based,  as 
noted  earlier,  on  the  general  systems  approach 
to  training  which  has  been  utilized  in  a 
variety  of  forms  in  the  training/simulation 
industry  over  the  past  20  years.  Whereas  the 
specific  content  of  the  guidelines  addresses 
the  masters-level  mariner  they  can  also  be 
applied  to  the  design  and  selection  of  training 
systems  for  other  levels  of  deck  officers.  The 
example  given  later  in  this  paper  addresses  the 
application  of  these  guidelines  and  the  actual 
design  of  a  simulator-based  training  system  for 
maritime  academy  cadets. 

The  use  of  ship  bridge/shiphandling  simulators 
is  growing  rapidly,  as  noted  above.  Each  of 
the  approximately  two  dozen  simulators  in  use 
world-wide  differ  substantially  from  every 
other  simulator/training  device.  These 
differences  generally  focus  on  the  visual 
scene,  although  they  also  pertain  to  other 
major  characteristics  such  as  the  hydrodynamic 
equations  of  motion.  As  a  result,  each 
simulator  has  substantially  different 
capabilities  and  limitations.  Furthermore, 
simulation  technology  is  improving  rapidly 
making  available  new  aoproaches  to  the  design 
of  simu^tor/training  device  characteristics. 
The  technology  advances  directly  relate  to  the 
cost/effectiveness  of  simulator-based 
training.  Although  the  maritime  industry  has 
had  experience  with  simulator/training  devices 
(i.e.,  part -task  radar  training  devices),  they 
have  had  little  experience  with  the 
sophisticated  and  complex  ship 
bridge/shiphandl ing  simulators  from  a  design 
and  applications  standpoint.  Hence,  the 


guidelines  have  been  developed  to  be  used  by 
shipping  companies,  labor  organizations,  and 
mariners  in  general.  The  guidelines  are  not 
specifically  intended  for  use  by  the  simulator 
manufacturer/electrical  engineer  in  the  sense 
of  providing  detailed  engineering  design 
characteristics  (e.g.,  number  of  pixels  on  the 
visual  screen).  Rather,  they  are  aimed  at  the 
operational  level  mariner.  They  are  intended 
to  give  him  guidance  with  regard  to  the 
characteristics  that  are  relevant  on  the 
simulator,  their  need  and  application  in 
training,  and  their  expected  capabilities  and 
limitations.  The  guidelines  translate 
operational  functional  requirements  into  their 
associated  training  system  characteristics. 

The  guidelines  user  (e.g.,  shipping  company 
training  specialist)  woula  use  the  information 
in  the  guidelines  to  (1)  specifically  detail 
his  particular  training  needs,  (2)  compare  his 
training  needs  with  the  capabilities  and 
limitations  of  each  alternative 
fidelity-quality  level  for  each  training  system 
characteristic,  and  (3)  arrive  at  a  conclusion 
to  specify  the  minimum  or  desired  level 
required  for  each  training  system 
characteristic  to  achieve  his  particular 
training  needs.  This  final  step  would  result 
in  a  set  of  training  system  functional  design 
characteristics  with  which  he  could  evaluate 
different  commercial ly  available  training 
system  facilities  for  purchasing  training  time, 
or  develop  a  specification  for  perspective 
bidders  to  develop  an  appropriate  training 
system. 

The  three  major  steps  of  the  guidelines,  which 
correlate  with  specific  sections  of  the  report, 
are  briefly  discussed  below. 

Training  Needs  (Section  »1) 

Training  needs  are  typically  identified  in  the 
form  of  operator  tasks  or  training  objectives 
that  are  tt  be  achieved  via  the  training 
program.  These  training  objectives  or  tasks 
must  be  feasibly  achievable  via  simulator-based 
training  on  a  cost/effective  basis.  A  specific 
set  of  masters-level  deck  officer  training 
objectives  (i.e.,  called  Specific  Functional 
Objectives)  that  are  cost/effecti-ely 
achievable  via  simulator-based  training  were 
developed  in  the  Phase  1  Investigation  of  the 
Training  and  Licensing  Project.  (■“O  Many  of 
the  traininq  objectives  in  this  set  can  also  be 
achievable  via  other  training  media  (e.g., 
at-sea  training).  The  guidelines  user, 
therefore,  would  use  this  set  of  training 
objectives  as  the  starting  point  for 
determining  his  specific  subset  of  training 
objectives.  Of  course,  he  may  also  add 
additional  training  objectives  not  in  the 
developed  set. 

To  select  the  specific  subset  of  training 
objectives  that  constitute  the  goals  of  a 
particular  course,  specific  criteria  must  be 
employed.  The  license  level  of  the  trainees 
(e.g.,  master)  is  one  such  criterion.  A  second 
criterion  is  the  type  of  training  that  will  be 
provided,  such  as  (1)  upgrading  to  a  higher 
license  level  (e.g.,  first  mate  upgrading  to 
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master);  (2)  training  to  handle  a  vessel 

substantially  different  than  those  in  which  the 
trainee  has  experience  (e.g.,  transitioning 

from  a  30,000  dwt  tanker  to  a  170,000  dwt 
tanker);  and  (3)  refresher  training  (e.g., 
periodic  retraining  in  emergency 

shiphandling).  Usually,  one  of  these  three 
training  types  is  specifically  addressed  in 
each  training  program.  A  third  criterion  is 
the  relative  importance  ol  improving  certain 

skills  (i.e.,  addressing  particular  training 
objectives).  The  guidelines  address  these 
criteria  in  the  form  of  recommendations  based 
on  analysis  of  marine  accidents. 

The  first  two  of  the  above  criteria  are 
relatively  straight-forward,  and  readilv 
specified  by  the  guideline  user.  To  provide 
uidance  with  regard  to  the  third  criterion 

i.e.,  training  areas  keyed  to  accident  causal 
factors)  seven  major  accident  analyses  were 
compiled  to  determine  the  major  areas  to 
receive  training  emphasis.  Six  major  training 
are ;s  resulted: 

1.  Navigation  management  training 

Pilot/master  relationship 
Bridge  procedures  (e.g., 
established  navigation  position, 
monitor  other  vessel) 

Bridge  organization  (e.g.,  vessel 
manning) 

2.  Ship-to-ship  communications  training 

3.  Shiphandling  training  (e.g.,  safe 
speed,  compensating  for 

external  forces) 


4.  Emergency  shiphandling  training  (e.g., 
power  failure-) 

5.  Rules-of-the-road  training 

6.  Restricted  waters  navigation  training 

If  the  regulatory  bodies  were  to  give  credit 
toward  a  license  for  simulator-based  training, 
such  credit  should  be  provided  for  training 
that  addresses  the  above  areas  (i.e.,  accident 
relevant  deck  officer  skills),  rather  than 
other  possible  areas  of  training  (e.g.,  ship 
productivity).  The  deck  officer  skills  that 
should  be  addressed  under  each  o*'  the  six  areas 
were  u.ntified,  to  provide  a  greater  depth  of 
guidance.  For  example,  the  following  is  one  of 
several  sets  of  skills  that  should  be  addressed 
under  Shiphandling  Training: 

"The  trainee  should  demonstrate  proficiency  in 

handling  a  specific  type  and  size  of  vessel  to 
stop  or  slow  the  vessel  effectively  under 
various  conditions  of  wind,  current,  and  water- 
depth  when: 

•  approaching  a  single  point 
mooring  buoy 

•  approaching  a  dock/pier 

t  maneuvering  to  bring  up  a  pilot 

•  maneuvering  to  bring  up  tugs 

t  anchoring" 

Additional  detail  regarding  ot.iW  skills  was 
provided  under  the  Shiphandling  I  raining  area, 
as  well  as  under  the  other  five  areas. 
Guidance  regarding  the  integration  of  the  above 
three  c.r : te  ia  is  provided  in  Table  1.  This 
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table  recommends  those  training  areas  (i.e., 
"H"  equals  high  priority)  that  should  be 
addressed  according  to  the  type  of  training 
(b.g.,  transition,  refresher,  upgrading)  and 
the  trainee  level  (e.o.,  masters-level ).  This 
table,  therefore,  recommends  those  training 
areas  that  should  be  addressed  with  regard  to 
the  guideline  user  supplied  criteria  (i.e., 
training  type  and  trainee  level).  The 
guideline  user  would  identify  his  training 
needs  by  entering  the  table  with  the  training 
type  and  trainee  level  c»  iteria,  and  receive 
guidance  with  regard  to  the  training  areas  that 
shou’d  be  addressed.  For  example,  transition 
training  programs  at  the  mas'er's  level  should 
address  Shiphandling  (#3)  and  Emergency 
Shiphandling  (#4)  training  areas. 

The  guidelines  themselves  provide  more 

detailed  information  req^ding  each  of  the  six 
training  areas.  The  guiatline  user  may  also 
have  other  criteria  to  further  delineate  his 
training  needs.  In  the  example  provided  later 
addressing  cadet  training  a  set  of  training 
objectives  were  developed  to  identify  those 
specific  training  needs  that  should  be 
achievable  via  simulator-based  training,  at-sea 
training,  and  classroom  training.  This  first 
step  of  the  guidelines,  therefore,  provides  the 
methodology  for  establishing  training  needs  and 
also  specific  criterion  information  in  ‘ nat 
regard. 

Training  System  Characteristics  (Section  f?) 

The  guidelines  address  the  design  and  use  of  a 
deck  officer  shiphandling  training  system, 
rather  than  me  ely  a  training  dev'ce  or 
simulator.  The  major  parts  of  the  training 
system  are: 

1.  Training  device  --  typically  viewed  as 
the  ship  bridge/shiphandling 

simulator;  but  actually  should  consist 
of  more  than  just,  the  simulator, 
including  feedback  displays, 

performance  indicators,  and  so  on. 

?.  Training  program  --  consisting  of  the 
detailed  training  course  curricul  , 
suoport  materials  (e.g.,  visual 
slides),  instructors  guide,  and  so  on. 

3.  Instructor. 

Tradi t ional 1 y,  the  training  system  emphasis  has 
been  on  the  design  of  simulator  (i.e.,  the 

real-world  fidelity  characteristics  of  the 
training  device).  Recent  research  has 
indicated  that  the  instructor  is,  perhaps,  the 
most  jmoortanf  element  of  the  training 
System. i*/  This  finding  indicates  that  the 
non-simulator  aspects  of  the  training  system 
are  at  least  as  important  as  the  simulator  with 
regard  to  achieving  an  effective  training 
process.  The  training  program  and  its 
associated  support  materials  represent  major 
tools  used  by  the  instructor  in  achieving  an 
effect  1 /e  training  process. 

Other  researcn  part  cula^ly  addressing  the 
training  device  has  indicated  that  “the 
training  device  is  more  than  a 


simulator". (7)  Rather,  the  training  device 
should  consist  of  both  simulation  and  training 
subsystems.  The  simulation  subsystem  consists 
of  those  fidelity  characteristics  attempting  to 
duplicate  real-world  functioning.  The  training 
subsystem,  on  the  other  hand,  provides  a 
variety  of  capabilities  to  assist  the 
instructor  in  conducting  an  effective  training 
process.  A  variety  of  instructor  support 
capabilities  (e.g.,  automatic  performance 
monitoring;  generation  of  graphical  feedback 
displays  to  illustrate  aspects  of  performance) 
are  readily  and  cheaply  available  on 
computer-based  simulator/training  devices. 
Research  has  indicated  that  these  capabilities 
can  have  a  substantial  impact  on  the 
effectiveness  of  simulator-based 

training. (S)  Hence,  the  cost/effective 
training  device  should  include  a  simulation 
subsystem  and  a  training  subsystem,  both  of 
which  should  be  tailored  to  the  specific 
aspects  of  the  training  needs  and  the  training 
situation  on  a  cost/effectiveness  basis. 

Specific  guidance  is  provided  for  each  of  the 
major  parts  of  the  training  system,  including 
the  s iu jl at  ion  and  training  subsystems  of  the 
training  device.  Critical  characteristics 
associated  with  each  of  the  major  parts  of  a 
simulator-based  training  system  have  beer, 
identified  (Table  2).  The  major  section  of  the 
guidelines  addresses  each  of  these  critical 
characteristics,  as  to1 lows: 

•  An  overview  of  each  critical 
characteristic  is  provided,  addressing 
available  alternative  technologies  for 
its  achievement,  and  pertinent  issues 
and  cons iderat ions  regarding  each 
characteristic. 

•  Three  levels  ot  fidelity/quality  nave 
been  established  for  each  critical 
characteristic.  Specific  guidance 
information  is  provided  regarding  the 
capabilities  and  limitations  of  each, 
level  as  pertains  to  the  behaviors, 
conditions,  ar.d  performance  standards 
associated  with  training  objectives. 

The  information  provided  in  the  guidelines  was 
derived  from  (1)  empirical  research  findings 
eminating  from  earlier  phases  of  the  Training 
-and  Licensing  Research  Program,  as  well  as 
other  independent  research  efforts;  and  {2} 
evaluation?  and  Judgements  by  operational  and 
training  experts.  Some  relative  costing 
information  is  also  included  where  pertinent. 
Specific  cost  information  in  a  set  guidelines 
such  as  these,  however,  would  likely  be 
inoccur ate  dee  to  the  rapidly  changing 
technology  or  tne  simulation  field.  Rather,  at 
the  time  of  applying  the  guidelines  specific 
costing  information  shouM  be  developed  isee 
below  with  regard  to  the  cadet  training  device 
design  application). 

A  specific  sample  of  the  format  «*nc  content  of 
the  g-. idelines  for  one  critical  characterist i . 
fi.e.,  simulator  design  <.har$<  tnrtstic,  visual 
scene,  horizontal  field  of  view)  is  provided  in 
Appendix  A.  This  example  is  illustrative  of 
the  level  of  detail  and  type  of  information 
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TABLE  2.  CRITICAL  SIMULATOR-BASED 
TRA I N INGSYSTEM  CHARACTERISTICS 


Simulator  Design  (Critical  C i aracteristics) 

Visual  Scene 

§  Geographic  Area 

0  Horizontal  Field  of  View 

0  Vertical  Field  of  View 

•  Time  of  Day 

§  Color  Visual  Scene 

§  Visual  Scene  Quality 

Radar  Presentation 

Bridge  Configuration 

Ownship  Characterist ics  and  Dynamics 

Exercise  Contrc1 

Traffic  Vessel  Control 

Training  Assistance  Technology 

Availability 

Training  Program  Structure _ { Cr i ♦  'r.al 

Characteristics) 


Skill  Levels  After  Trailing 
Skill  Levels  Prior  to  Training 
Training  Objectives 
Training  Techniques 
0  Knowledge  of  requirements 
0  Positive  guidance 
0  Adaptive  training 
0  Post  problem  critique 
Instructor's  Guide 
Classroom  Support  Material 
Simulator/Classroom  Mix 
Training  Program  Duration 
Class  S  ze 
Scenario  Design 
Number  of  Scenarios 
Stress 

Over ’earning 

Instructor  Qualifications _ (Critical 

Characterist ics) 

Mariner  Credentials 
Instructor  Credentials 
Subject  Knowledge 
Instructor  Skil-S 
instructor  Attitude 
Student  Rapport 
Instructor  Evaluation 


contained  in  this  sect;on  of  the  guidelines. 
Information  similar  to  tnat  in  Appendix  A  is 
provided  'or  the  other  critical  design 
character ist ics  of  the  simulator,  the  training 
program  structure,  and  the  instructor 
qua! if ications.  As  might  be  expected, 
relatively  little  detailed  objective 

information  is  available  from  the  research 
literature  regarding  any  of  the  above  training 
system  parts.  The  Training  and  Licensing 
•Voieet  concluded  mat  the  empirical  data 
r*  .ulting  from  the  research  on  CAORF  supports  a 
nn;hly  structured  subjective  approach  to  the 
design  of  the  training  system.  This,  in 
essence,  supports  the  approach  taken  in  these 
design  Guidelines,  combining  available 

objective  data  as  baseline  information,  and 


using  highly  structured  subjective  evaluations 
to  address  the  remaining  necessary 
characteristic:.  Following  this  approach, 
considerable  infonnotion  has  been  developed  in 
the  guidelines  regarding  the  simulator  and  the 
training  program.  Relatively  little 
information,  however,  is  provided  in  the 
guidelines  for  the  instructor  characteristics. 
This  is  due  to  a  general  lark  of  investigation 
regarding  the  qualities  of  an  effective 
instructor. 

Recomuendations  For  Master-Level  Mariners 
(Section  HT 

The  generic  part  of  the  guidelines  consists  of 
the  above  two  sections.  The  generic  structure 
is  complimented  oy  specific  information  content 
3Cdressing  the  masters-level  deck  officer.  The 
third  and  final  section  of  the  guidelines  is 
not  generic,  as  is  the  above  two  sections,  but 
r/ther  provides  recommendations  specific  to  the 
masters-level  deck  jfficer.  The  guide". ine  user 
reeds  are  thus  integrated  with  the  training 
system  characteristics  guidance  information  to 
achieve  a  specific  set  of  recommendations.  The 
recommendations  address  the  six  training  areas 
and  the  masters-level  license;  ihey  do  not, 
however,  address  the  thre:>  training  tjpes. 
They  could,  of  course,  be  further  reiinea  to 
address  each  of  the  training  types  (e.g., 
transition  training). 

The  recommendations  provide  a  "Recommended 
Level"  and  a  "Minimum  Level"  associated  with 
each  of  the  critical  training  system 
character istics,  for  each  of  the  six  training 
areas.  Table  3  lists  the  Recommended  and 
Minimum  levels  for  the  training  system 
characteristics  associated  with  the  Emergency 
Shiphandling  training  area.  If  the  guideline 
user  was  going  to  purchase  a  simulator-Dased 
training  program  for  emergency  shiphandling 
training  of  master-level  deck  officers,  he 
would  go  to  this  table  to  determine  the 
requisite  levels  of  critical  training  system 
characteristics.  Several  examples  from  laole  3 
are: 

0  The  recommended  simulator  visual  scene 
horizontal  field  of  view  should  be 
greater  2VJ  degree ;  the  a.nimum 

acceptable  level  would  be  120  degrees 
to  240  degrees  inote,  th«  se  refer  to 
specific  categories  addressed  in 
Section  »2  of  the  guidelines). 

0  The  : 'lining  program  duration  is 

recc  va*  ided  as  3  days;  this  is  also 
considered  to  be  the  miritcum  training 
program  duration. 

0  The  training  program  class  size  is 
recommended  as  3  students  or  less; 
however,  up  to  b  Students  would  be 
acceptable;  greater  than  b  students  in 
a  class  would  be  unacceptable. 

0  ?  ii  recommended  that  the  instructor 

fc.sess  a  pilot's  license  or  pilot 
endorsement;  he  should,  as  a  minimum, 
have  a  masters  license. 
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SLUM ARY  or  RECOMMENDED  TRAINING  SYSTEM  GUIDELINES 
FOR  EMERGENCY  SHIPHANDLING 


TA31.C  3. 

CRITICAL  TRAINING 
SYSTEM  CHAR A CTtM I  ST  I CS 

Simulator 

Visual  Scene 

Geographic  area 
Horizontal  FOV 
Vertical  FOV 
Time  of  day 
Color  visual  scene 

Radar  Presentation 

Bridge  Ccnf  i (jurat  icn 

Owns!) ip  Character i st i cs 

Exercise  Control 

Traffic  Vessel  Control 

Training  Assistance  Technology 

-  Availability 

Training  Program 

Skill  Level  After  Training 

Skill  Level  Before  Training 

Training  Objectives 

Training  Techniques 

Knowledge  of  requirements 
Positive  guidance 
Adaptive  training 
Post  problem  critique 

Instructor's  Guide 

Classroom  Support  Material 
Simula tor/ Classroom  Mix 

Training  Program  Duration 
Class  Si*c 
Scenario  Design 
Number  of  Scenarios 
Stress 

Overlearning 

Instructor 

Mariner  Credentials 
Instructor  Credentials 
Subject  Knowledge 
Instructor  Skills 
Instructor  Attitude 
Student  Rapport 
Instructor  Evaluation 


RECOMMENDED  LEVEL 

III:  Restricted  water 
III:  Greater  than  240° 

III:  Greater  than  *15° 

III:  Day/night  ~ 

II:  Multi -color 

II:  Low  fidelity  radar 

II:  Full  bridge 

III:  Special  effects 

III:  instructor  exercise  control 

III:  Indepe  >ent1y  maneuverable 

II:  Feedback  display 

II:  High  availability 

II:  Direct  skill  Improvement 
III:  Simulator  diagnostic  evaluation 
III:  Highly  structured 

Various  techniques 
Various  techniques 
II:  Group  adaptive  training 
III:  Complete  and  Immediate  feedback 

III:  Documented  instructor's  guide 

III:  Advanced  support  media 
III:  Prebrief ing/simulator/post- 

briefing  mix 
II:  3  Days  (24  hours) 

I:  3  or  less  students 
Various  levels 
III:  Desired  practice 
III:  Progressive  stress 
II:  Desired  overlearning 

III:  Pilot's  license/endorsement 

III:  Instructor  course 

II:  Exhaustive  knowledge 

III:  Outstanding 

III:  Enthusiastic 

II:  Respected 

I':  Continuing 
II:  Diagnostic 


minimum  level 

II:  Coastal 
II:  1 20°  to  240° 

II:  -HO0  to  *15° 

1 :  I)ay  only 
I:  Black  and  white 

I :  No  radar 

I:  Reduced  bridge 

1 1 :  Shal low  water 

I:  Exercise  selection 

I:  Canned  traffic 

NONE 

II:  High  availability 

II:  Direct  skill  improvement 
I:  No  diagnostic  evaluation 
II:  Moderately  structured 

Various  techniques 
Various  techniques 
1:  No  adaptive  training 
II:  Immediate  feedback 

II:  Undocumented  instructor's 
guide 

I:  Basic  support  material 
II:  T'imulator/postbrlef ing  mix 

II:  3  Days  (24  hours) 

II:  6  or  less  students 
Various  levels 
II:  Moderate  practice 
III:  Progressive  stress 
I:  No  overlearning 

II:  Master  license 

I:  Educational  certificate 

I:  Satisfactory  knowledge 

I:  Marginal 

I:  Reserved 

I:  Competent 

I:  Continuing 
II:  Diagnostic 
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The  recommended  end  minimum  levels  associated 
with  each  of  the  critical  training  system 
characteristics  are  the  titles  pertaining  to 
the  alternative  levels  of  f idei ity/qual ity  for 
each  of  the  respective  characteristics  (Section 
#2).  Note  that  the  general  ranking  of  the 
alternative  levels  are  also  indicated  ( i . e . ,  1 
lowest  -  III  highest).  The  guideline  user 
would  obviously  refer  to  Section  *2  when  using 
this  table  of  recommendation*. 


Guideline  Summary 


The  guidelines  for  deck  officer  training 
systems  has  three  major  parts:  (1)  training 
needs,  (2)  training  system  characteristics,  and 
(3)  pecific  recommendations  for  masters-level 
deck  officers.  The  structure  of  the 
guidelines,  represented  by  the  first  two 
sections,  is  generic  and  is  applicable  to 
various  levels  of  deck  officers,  types  of 
training,  and  training  objectives.  The  content 
of  these  sections  of  the  guidelines  are 
specific  to  the  masters-level  deck  officer, 
although  they  do  generally  apply  to  other 
levels  of  deck  officers.  The  third  section  of 
the  guidelines  represents  the  integration  of 
the  first  two  sections  pertaining  specifically 
to  the  masters-  level  de'k  officer;  it  is  hence 
not  generic. 


The  remainder  of  this  paper  discusses  an 
example  of  the  application  of  these  guidelines 
to  the  design  of  a  simulator-based  training 
ievice.  The  particular  training  oevice 
designed  was  not  for  masters-level  training,  as 
are  the  above  examples.  Rathe* ,  it  was  for 
cadet-level  training.  Hencev  the  general 
structure  of  the  guidelines  was  used,  although 
the  specific  content  had  to  be  developed  and 
tailored  vor  the  cadet-level  training. 


APPLICATION  of  training  system  design  guidelines 


Background 


In  1980  the  Maritime  Administration,  realizing 
the  potential  benefits  of  simulator  training 
within  the  maritime  cadet  curriculum,  directed 
CAORt  to  develop  (1)  a  functional  specification 
for  a  maritime  academy  ship  bridge/shiphandl  ing 
simulator  and  (2)  training  program 
recomendat ions  for  the  proper  integration  of 
such  simulator  training  into  the  maritime 
academy  program.  This  developmental  work  was 
conducted  as  part  of  the  Training  and  Licensing 
Project.  It  urew  heavily  on  the  *' guidelines 
for  Deck  Officer  Training  Systems"  and 
represents  the  first  practical  application  of 
the  guidelines.  It  shoulc  be  noted  that 
simulator-based  training  is  not  an  unfamiliar 
training  medium  for  the  academies  since  all 
have  radar  simulators  and  some  have  cargo 
handling  simulators.  However,  none  of  the  six 
academies  have  a  full  mission  ship 
bridge/shiphandl ing  simulator,  nor  do  they  have 
experience  with  the  operation  of  this 
potentially  powerful  training  device  which  may 
be  many  times  as  complex  .is  the  normal  radar 
s  irtu’  ator. 


The  approach  taken  to  acnieve  these  objectives 
was  consistent  with  the  overall  appi  oach  and 


findings  throughout  the  Training  and  Licensing 
Project,  which  base  the  design  and  utilization 
of  a  training  device  on  the  specific  SnIUs  to 
be  achieved.  The  analysis  specifically 
addressed  the  ru*l  1  -miss  ion  ship 
bridge/shiphandling  simulator.  Although  other 
training  media  were  considered  with  regard  to 
achievement  of  the  identified  training 
objectives,  the  resultant  characteristics  were 
determined  only  for  the  ship 
bridge/shiphandling  simulator.  The  specific 
approach  taken  involved  the  following 
sequential  steps: 

1 .  Identification  of  Maritime  Cadet 
Training  Objectives.  A  listing  of  specific 
mariti me  cadet  training  objectives  were 
identified,  based  on  third  mate  watchstanding 
tasks  along  with  the  skill  and  knowledge 
requirements  for  proficiently  performing  these 
tasks.  It  should  be  noted  that  these  represent 
the  desired  skills  and  knowledge  of  a  new  third 
mate  upon  graduation  from  a  maritime  academy 
and  entry  into  the  merchant  marine,  for 
watchstanding  tasks  only.  Tne  training  and 
education  provided  By  t :he  various  maritime 
academies  is  obviously  much  broader  than  those 
identified  for  this  analysis  of  snip 
bridge/shiphandl  ing  simulator-based  training. 
Fne  developed  training  objectives  formed  the 
behavioral  data  base  of  training  naeus. 

2.  Analysis  of  Maritime  Cadet  Training 
Object  ive?!  The  identified  cadet  training 
objectives  were  tnen  allocated  to  various 
training  media  available  for  maritime  cadtt 
training  (i.e.,  classroom,  small  vessel,  at-sea 
training  ship,  or  simulator).  Those  training 
objectives  which  were  identified  as  being 
achieved  best  via  simulator  and  were  of  highest 
priority  were  designated  as  the  goals  of  tue 
simulator-based  training  system  design  process. 

3.  Establishment  of  Simulator  Functional 
Requirements^  Primarily  using  the  information 
contained  Tr  the  "Guidelines  for  Deck  Officer 
Training  Systems",  the  functional  requirements 
for  a  maritime  cadet  simulator  were  developed 
to  meet  the  identified  training  oDjectives. 
Appropriate  consideration  was  given  to  both  the 
cost  and  benefits  associated  with  the  various 
levels  of  each  simulator  characteristic  (e.g., 
time  of  day:  day  vs  night). 


4 . 

Recomr? 

requirements  for  the  cadet  simulator  were 

established,  recommendations  for  the 

integration  of  this  type  of  training  into  the 
academics'  training  programs,  and  the 
qualifications  for  the  instructor,  were 

developed.  These  recommendations  were  also 

b.sed  primarily  on  the  informat  ion  contained  in 
the  "Guidelines  for  Deck  Of; icer  Training 
Systems". 


Development 

lendations. 


training 

the 


Program 

functional 


Tr  above  four  steps  follow  directly  from  tne 
guidelines.  Whereas  the  training  needs  in  the 
guidelines  were  developed  from  masters-level 
training  objectives  and  an  analysis  of  shipping 
accidents,  t.ose  for  cadets  were  based  on  an 
independent  analysis.  The  specific  guidance 
information  in  the  guidelines  was  used  heavily 
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sin<"‘  it  is  generic.  The  guideline 
recommendations  (e.g.,  Table  3)  were  generally 
not  used  since  they  are  specific  to  the 
master's  level.  Rather,  independent 
recommendations  were  arrived  at  specific  to  the 
cadet-level.  The  remainder  of  this  paper 
addresses  the  findings  and  recommendations  for 
the  cadet  ship  briage/shiphand ling 
simulator-based  training. 

General  Findings 

The  majority  of  the  identified  cadet  training 
objectives  can  be  trained  at  sea,  although  this 
may  not  be  the  most  cost  effective  training 
medium  for  many  of  the  desired  skills.  Many  of 
the  cadet  training  objectives  can  be 
effectively  taught  only  at  sea,  while  others 
are  best  t~ai  ed  at  sea  although  other  media 
would  also  be  effective  Finally,  a  subset  of 
the  training  objectives  were  found  to  be 
best-trained  via  a  ship  bridge/shiphandling 
simulator.  These  judgements  were  arrived  at  on 
the  basis  of  the  simulator  capabilities  and 
limitations  provided  in  the  guidelines,  and  the 
judgements  of  mariner  and  training  experts. 
Note  that  cost  was  not  a  detailed  consideration 
at  this  stage  of  analysis,  but  was  brought  in 
later  when  trade-offs  were  made  regarding 
alternative  fidelity  levels  of  simulator 
characteristics. 

Recommendations  For  Simulator  Characteristics 

Fourteen  critical  characteristics  were 
identified  for  the  functional  design  of  a 
maritime  cadet  ship  bridge/shiphandling 
training  device.  A  specific  level  for  each  of 
these  cnaracteristics  has  been  recommended  as  a 
minimum  for  meeting  the  needs  of  cadet 
training,  on  the  basis  of  their  cost  and 
training  effectiveness.  A  brief  explanation 
and  rationale  summary  for  selection  of  each 
level  of  these  critical  characteristics 

follows.  Considerably  more  rationale  and 
descriptive  information  on  these  functional 
simulator  characteri sties  is  contained  in  the 
project  report. (6)  (Note  that  the  relative 
cost  factors  given  below  arj  associated  with 
particular  training  device  subsystems,  not  the 
overall  device  cost. 

•  Visual  scene  time  of  day  --  Night  --  much 

of  tfie  requisite  caclet  training  could  be 
achieved  under  either  night  or  day 
conditions;  the  night  capability  is 
estimated  to  cost  about  2.5  times  less  than 
the  dciy/night  capability;  the  academies 

have  a  limited  amount  of  time  available, 
and  nighttime  is  the  more  difficult 

condition. 

«  Visual  scene  geographic  area  --  Coastal  -- 

tRe  majority  of  RTghly  critical 
simulator-best  training  objectives  for 
cadets  do  not  require  shiphandling  skill  in 
restricted  waters;  cost  is  substantially 
less  fo-  coa.tal  (i.e.,  about  40  percent 
less). 

•  Visual  scene  horizontal  field  of  view  — 

180  Degrees  “  adequate  horizontal 

separation  of  geographic  bjects  across  a 


sufficient  number  of  training  exercises  is 
necessary  to  ensure  proper  development  and 
general izabi 1  it/  of  the  associated  visual 
position-fixing  skills;  will  cover  all  the 
critical  meeting  and  fine  crossing 
situations  called  for  by  training 
objectives;  and  is  necessary  for  coastal 
navigation/pi  luting  skills. 

•  Visual  scene  vertical  field  of  view  --  20 
Degrees  --"  several  critical  training 
objectives  require  a  moderate  vertical 
tield  of  view  to  handle  close-in  traffic 
vessels;  a  large  vertical  field  of  view  is 
unnecessary  in  a  nighttime  situation  since 
the  upper  and  lower  bounded  edges  would  be 
unnoticeable. 

•  Visual  scene  color  --  Multicolor  --  a 
night-only  vTsuTT  scene  should  have 
multicolor;  research  indications  are  that 
color  is  desirable  for  nigh  workloads;  the 
additional  cost  for  multicolor  under 
nighttime  conditions  may  not  be  substantial. 

•  Visual  scene  quality  --  Moderate  Quality  -- 

this  characteristic  depends  upon  the 

interaction  of  many  parameters,  each  of 

which  could  vary  widely  and  oe  acceptable 
depending  upon  the  level  of  the  other 
interacting  parameters  (e.g.,  brightness 
and  contrast  ratio);  the  complexity  of  the 
visual  scene,  the  large  number  of  relevant 
parameters,  and  the  lack  of  definitive 
research  information  precludes  detailed 

specification  at  this  time;  rather, 
specific  proposed  visual  scenes  snoJd  be 
evaluated  for  their  quality  at  the  time  of 
proposal  evaluation;  guidance  principles 

for  evaluating  visual  scene  quality  are 
included  in  the  project  report. 

•  Radar  presentation  --  Low  Fidelity  --  the 

acceptable  low  fidelity  raaar  would  be  a 
real-time  updated  picture  generated  by  a 
general  purpose  computer-display  system, 
with  the  display  located  in  the  wheelhouse 
in  place  of  a  commercial!;,  available  radar 
unit;  low  fidelity  radar  presentation  would 
be  satisfactory  for  achievement  of  nearly 

all  highly  critical  training  objectives; 
high  fidelity  rador  would  require  a  nearly 
four-fold  increment  in  cost;  the  academies 
have  high  fidelity  raaar  simulators 
presently  for  part-task  training. 

•  Bridge  conf iguration  --  Full  Bridge  --  the 

full  bridge  would  consist  of  a  normal  pilot 
house  layout  with  appropriate  equipment  and 
instruments;  adequate  bridge  size  is 
required  to  handle  the  anticipated  bridge 
team;  a  reduced  bridge  size  may  result  in 
irregular  third  mate  behavior  due  to  an 

abnormal  pilothouse  layout;  detailed 
functional  requirements  for  bridge 
equipment  are  contained  in  the  project 
report. 

•  Ownship  characteristics  and  dynamics  -- 
Deep  Water  --  deep  water  is  sufficient  for 
training  the  majority  of  the  highly 
critical  training  objectives;  the 
additional  cost  for  shallow  water  effects 
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does  not  appear  warranted  in  view  of  the 
minimal  training  objectives  gained; 
additional  hydrodynamic  requirements  are 
specified  in  the  project  report. 

•  Exercise  control  --  Instructor  Exercise 
Control  ~  this  level  enables  shipboard 
casualties  and  other  conditions  (e.g., 
wind,  current)  to  be  controlled  in 
real-time  from  an  instructor/operator 
console,  rather  than  have  all  aspects  of 
the  problem  always  preprogrammed; 
substantial  y  greater  training  flexibility 
at  a  relatively  small  increase  in  cost. 

•  Traffic  vessel  control  --  Independently 

Maneuverable  Traffic  --  necessary  for 
interaction  between  ownship  and  traffic 
vessels,  particularly  intership 

communications  (e.g.,  radiotelephone);  this 
woulJ  be  a  very  minor  increase  in  cost 
above  the  lower  fidelity  levels  for  this 
characteristic  in  several  simulation 
technologies. 

•  Training  assistance  technology  --  Remote 
Monitoring  --  displays  and  readouts  placed 
in  a  classroom  to  enable  a  t'oup  of 
students  to  monitor  and  discuss  the 
scenario  situation  and  ^tivities  of  the 
bridge  team  while  the  problem  is  in 
progress;  would  enable  the  simultaneous 
training  of  multiple  bridge  teams;  research 
has  shown  this  to  be  a  highly  desirable 
capabi 1 ity. (4) 

•  Training  assistance  technology  --  Feedback 
Di  splays  ~  a  display  located  in  the 
classroom  presenting  detailed  information 
concerning  the  just  completed  simulator 
exercise;  enables  a  variety  of 
training/investigative  activities  to  take 
place  in  the  classroom;  research  has  shown 
this  to  greatly  increase  the 
cost-effectiveness  of  training. (8) 

•  Avail abi 1 ity  --  the  simulator  design  goal 
is  for  operational  training  to  be  conducted 
thirty  hours  per  weex  with  95  percent 
availability;  an  additional  10  hours  per 
week  was  allotted  to  maintenance  time; 
vendor  support  of  the  simulator  was 
recommended  with  assistance  from  an  academy 
staff  technician. 

Training  Device  Cost 


To  estimate  the  likely  cadet  training  device 
costs,  the  developed  functional  requirements 
were  reviewed  by  several  individuals  who  were 
recognized  as  knowledgeable  in  simulator 
design.  Based  on  their  input  the  following 
cost  estimates  were  developed  for  the 
recommended  training  device. 

•  Initial  System  Procurement 

Lowest  possible  cost  =  $1.5M 
Highest  possible  cost  =  J3.5M 
Most  Like!/  cost  =  J2.7M 

•  Annual  Operating/Maintenance 


Lowest  possible  cost  =  $180k 
Highest  possible  cost  =  $320K 
Most  Likely  cost  =  $220K 

The  above  figures  are  provided  in  1982 
dollars.  They  assume  that  a  suitable 
building  exists  at  the  specific  maritime 
academy  to  house  the  simulator  facility. 
They  also  assume  that  the  single  instructor 
required  for  training/system  operation  and 
the  single  technician  required  for 
maintenance  will  be  obtained  from  the 
academy's  staff. 

Training  Program  Recommendations 

Training  program  recommendations  were  made 
to  assist  in  the  integration  of  a  ship 
bridge/shiphandling  siml ator-based  training 
program  into  the  academies'  curricula.  It 
is  essential  that  the  training  device  is 
accompanied  by  appropriate  supporting 
training  materials,  guidance  information 
concerning  its  rse,  and  application 
assistance  when  provided  to  the  academies. 
In  order  to  establish  a  common  basis  tor 
the  effective  integration  of 

simulator-based  training  into  the  maritime 
academy  curriculum,  the  curricula  of  the 
state  academies  were  compared.  As  a  result 
of  th i ^  corpirison  a  "umber  of  observations 
were  maue  which  form  the  basis  for  the 
training  program  recommendations  contained 
in  the  project  repurt.  The  following  is  a 
summary  of  these  observations : 

•  Each  of  the  state  maritime  academies 
appear  to  have  four  distinct  training 
periods  within  their  curricula.  These 
training  periods  are: 

From  academy  entry  to  first  at-sea 
period 

From  first  at-sea  period  to  second 
at-sea  period 

From  second  at-sea  period  to  third 
at-sea  period 

From  third  at-sea  period  to 
graduation 

•  Each  academy  offers  a  radar  observer 
course  utilizing  its  radar  simulator 
training  facility.  This  course  is 
usually  given  in  the  junior  (2nd  class) 
year,  although  one  academy  offers  the 
course  to  seniors  (1st  class). 

•  Several  academies  have  indicated  concern 
relating  to  the  integration  of 
additional  simulator  training  into  the 
already  intensive  cadet  schedule. 

After  careful  consideration  of  the 
similarities  identified  above  and  with  due 
respect  for  the  individual  state  academy's 
ability  to  determine  the  proper  means  of 
integrating  ship  bridge/shiphandling 
simulator  training  into  their  own 
curriculum,  appropriate  recommendations 
were  made.  The  following  is  a  summary  of 
the  training  program  recommendations 
contained  in  the  project  report. 
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SUMMARY 


•  The  academies  should  consider  grouping 
the  training  objectives  previously 
identified  into  four  training  modules 
which  are  described  in  the  project 
report. 

Module  #1:  Basic 

Watchstanding. . .prior  to 
first  at-sea  period 
Module  #2:  Coastal 

Navigation. . .prior  to 
second  at-sea  period 
Module  #3:  Collision 

Avoidance. . .after  radar 
simulator  course,  prior 
to  third  at-sea  period 
Module  #4:  Advanced 

Watchstanding. . .prior  to 
graduation 

•  Each  training  module  should  consist  of  a 
series  of  simulator  exercise  periods, 
each  approximately  3  hours  in  duration. 
The  individual  academy  staff  should  have 
the  option  to  either  (a)  integrate  these 
simulator  exercise  periods  into  the 
existing  course  as  laboratory  periods  or 
(b)  provide  all  the  simulator  exercise 
periods  with  each  training  module  as  a 
new  course. 

•  The  simulator  training  should  be  related 
to  the  types  uf  tasks  that  the  cadet 
will  be  performing  during  the  next 
at-sea  period.  For  example.  Module  #1 
should  precede  the  first  at-sea  training 
period. 

•  A  carefully  structured  training  program 

should  be  employed  for  maritime  cadet 
training.  Tne  project  report  provides 
appropriate  guidance  for  the  following 
critical  training  program 

characteristics  as  specifically  applied 
for  cadet  training: 

Training  Objectives 
Training  Techniques 
Instructor's  Guide 
Classroom  Support  Materials 
Simulator/Classroom  Mix 
Training  Program  Duration 
Class  Size 
Scenario  Design 

•  The  instructor  is  extremely  critical  for 

effective  training.  The  project  report 
lists  and  discusses  the  following 
critical  instructor 

qualifications/characteristics  to  assist 
in  the  proper  selection  and  preparation 
of  maritime  cadet  instructors: 

Mariner  Credentials 
Instructor  Credentials 
Nautical  Science  Knowledge 
Instructor  Skills 
Instructor  Attitude 
Stude  it  Rapport 
Instructor  Training 
Number  of  Instructors 
Instructor  Evaluation 


The  availability  of  simulation  technology 
and  the  use  of  ship  bridge/shiphandling 
simulators  for  the  training  of  merchant 
marine  deck  officers  has  rapidly  taken 
place  during  the  past  decade.  This  has 
evolved  to  fill  a  need  created  by 
world-wide  increase  in  shipping,  shipping 
casualties,  and  the  consequences  of  these 
accidents.  The  Training  and  Licensing 
Project  was  initiated  by  the  U.S.  Coast 
Guard  and  U.S.  Maritime  Administration  to 
define  the  role  of  the  ship 
bridge/shiphandling  simulator  in  the  deck 
officer  Training  and  Licensing  process.  A 
major  product  eminating  from  this 
multi-year  project,  culminating  a 
substantial  amount  of  research,  is  a  set  of 
guidelines  for  the  design  of  deck  officer 
training  systems.  These  guidelines  are 
aimed  specifically  at  tne  user  of 
simulator-based  training  in  the  maritime 
industry  (e.g.,  shipping  company,  labor 
organization).  The  guidelines  are  written 
at  a  level  to  be  used  by  individuals  making 
major  decisions  regarding  the  use  of 
simulator-based  training  programs,  who  are 
often  not  familiar  with  the  engineering  and 
training  technologies  associated  with 
simulator-based  training  systems,  and  who 
are  concerned  with  the  achievement  of 
operational  training  goals.  The  guid_lines 
provide  a  generic  structure  that  is 
applicable  across  various  deck  officer 
license  levels,  training  types  (e.g., 
transition,  refresher,  upgrading)  and  other 
necessary  criteria  established  by  the 
user.  The  guidelines  establish  specific 
training  needs.  In  the  context  of  the 
masters- level  deck  officer  for  which  the 
guidelines  were  initially  developed,  the 
training  needs  aie  based  on  (1)  training 
objectives  amenable  to  simulator-based 
training,  and  (2)  high  priority  areas  of 
human  performance  as  determined  from 
analysis  of  shipping  accidents.  The  three 
major  parts  of  the  training  system  are 
addressed,  including  (1)  the  training 
device,  consisting  of  a  simulator  and  other 
instructional  aids;  (2)  the  training 
program,  and  (3)  the  instructor.  Critical 
characteristics  are  identified  for  each  of 
these  training  system  parts;  three  levels 
of  fidelity  or  quality  are  identified  for 
each  of  the  characteristics.  Guidance 
information  regarding  the  available  design 
technologies,  their  capabilities  and 
limitations,  and  other  issues  are  provided 
concerning  each  of  the  alternative 
fidelity/quality  levels  for  each  training 
system  characteristics.  Finally,  a  se*  or 
specific  recommendations  are  mede  for  the 
design  of  a  training  system  to  meet  masters 
level  training;  these  are  specified  in 
terms  of  a  recommended  level  and  a  minimum 
acceptable  level  for  each  training  system 
characteristic ;  the'  training  system 
characteristics  are  further  recommended 
separately  for  each  of  the  six  major  areas 
of  training. 
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An  applied  example  of  the  application  of 
the  guidelines  is  U'scussed.  This 
application  was  devoted  to  the  development 
of  a  functional  specification  tor  a  ship 
b^idge/shiphandl ing  simulator  for  the 
training  of  mdritime  academy  cadets.  This 
initial  application  of  the  guiue-iines 
addresses  not  only  the  design 
characteristics  of  the  training  device,  but. 
also  the  development  of  guidance  material 
for  use  by  the  maritimp  academies  for 
integration  of  the  simulator-based  training 
into  their  curriculum,  and  for  the 
development  of  an  effective  training 
program  to  support  the  simulator-based 
training. 

APPENDIX  A.  EXAMPLE  GUIDELINES  CONTENT  FOR 
SIMULATOR  DESIGN  CRITICAL  CHARACTERISTIC, 

VISUAL  SCENE,  HORIZONTAL  FIELD  OF  VIEW 

Horizontal  Field  of  View 

The  horizontal  field  of  view  required  for  a 
sh  iphanol ing/navigaticn  simulator  should 
depend  on  the  specific  objectives  of  the 
training  program.  if  the  visual  cues 
required  to  execute  a  particular 
shiphandling  maneuver  are  within  a 
relatively  narrow  field  of  view;  such  as 
when  training  the  skill  of  utilizing  range 
lights,  a  reduced  field  of  view  is 
satisfa  .t.>ry  and  may  even  be  preferable 
since  it  artificially  focuses  the  trainee's 
attention  on  the  required  visual  cues. 
However,  prudent  training  practice  would 
indicate  that  the  student  should  then  oe 
trained  in  utilizing  this  skill  under 
conditions  with  operational  noise  and 
distractions;  tor  example,  identifying  J.e 
range  lights  and  concentrating  on  them 
among  the  background  lights  and  distracting 
traffic  vessel  movement.  This  type  of 
trail  ing  could  then  imply  a  requirement  for 
greater  horizontal  field  of  view  than  that 
identified  for  the  development  of  the  basic 
skill.  Consideration  should  also  be  given 
to  the  utilization  of  a  variable  horizontal 
field  of  view  in  order  to  gain  the  training 
leverage  discussed  above. 

The  cost  of  a  ship  bridge/shiphandling 
simulator  increases  as  the  horizontal  field 
of  view  increases.  This  increase  in  cost 
results  not  only  from  increased  projection 
equipment  costs  but  also  from  increased 
processing  hardware  and  software  costs. 
This  is  particularly  true  tor 
computer-generated  graphic  systems. 

Level  I:  Greater  than  9(J°,  less  than 
1  2  'P~.  CF"G  of  this  horizon!  al  field  oT 
view  may  be  satisfactory  for  t.raininq  a 
limited  number  of  specific  shiphandl ing 
skills  (e.g.,  range  lights,  buoyed 
channels).  It  may  also  be  satisfactory  tor 
training  the  application  of  the  rules  ot 
the  road  in  meeting  and  fine  crossing 
situations.  However,  if  it  is  employed  in 
broader  crossing  situations  ir  overtaking 
situations  whore  visual  contact  is  lost 
with  the  traffic  vessel,  there  may  he  a 
danger  thn  the  trainees  will  have  a 


tendency  to  neglect  visual  bearings  and 
rely  heavily  on  radar  in  these  types  of 
scenarios.  A  horizontal  field  of  view  of 
less  than  120  degrees  is  generally 
unacceptable  for  training  skills  that 
involve  visual  position  fixing  sir,:e 
adequate  horizontal  angular  separation  of 
suitable  geographic  points  suitable  for  a 
visual  fix  can  not  be  obtained  except  for 
possibly  a  few  unique  cases.  In  this  same 
light,  such  a  limited  horizontal  field  of 
view  also  precludes  the  development  of 
skills  in  the  use  of  turn  bearings.  There 
may,  however,  be  some  training  value  for  a 
horizontal  field  of  view  of  less  than  120 
degrees  in  the  dc/elopmen"  of  skills 
involving  the  integration  of  visual  I  mes 
of  position  with  radar  information  or  other 
electronic  navigation  information,  although 
the  trainee  may  be  inadvertently  trained  to 
neglect  the  more  advantageous  objects  abeam 
for  visual  bearings. 

Level  II:  Greater  than  1^0°,  less 
than  240°.  Use  of  tins  horizontal  field 
o?  view  appears  appropriate  "or  the 
majority  of  the  desired  skills  categories 
identified  earlier.  It  may,  however,  be 
limited  if  visual  bearings  abaft  +  120 
deqrees  relative  are  import jnt  for 
navigation  in  a  particular  port.  In 
addition,  the  application  of  the  rules  ot 
the  road  in  an  overtaking  situation  is  also 
constrained,  although  only  for  the 
situation  when  ownship  is  being  overtaken 
and  not  when  ownship  is  doing  the 
overtaking.  This  situation,  however,  is 
somewhat  unique  and  not  particularly 
difficult  ( i . e . ,  requiring  specific 
training)  since  it  usually  involves  a 
relatively  slow  closing  rate  which  allows 
substantial  time  for  analysis  and  action. 


Level  III: 
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•  Use  ot  visual  bearings  abaft  ♦_  UO 
degrees  relative  (e.g.,  specific 
port  requirement) 

It  should  be  noted  that  many  ot  t ho  visual 
scene  generating  technologies  have  the 
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opt ica 1 ly/elec tron ic a  1  ly  roliting  the  rived 
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of  visual  bearings,  or  when  approaching  and 
picking  up  a  tow.  This  flexibility  with 
the  simulated  visual  scene  should  be  used 
cautiously  since  it  alters  the  bridge 
environment's  proper  orientation  with  the 
visual  scene  (i.e.,  front  of  pilothouse 
faces  side  of  vessel).  The  impact  of  this 
effect  on  the  training  provided  is  unknown. 


6.  Gynther,  J.  W.,  T.  J.  Hammell,  J.  A. 
Grasso,  and  V.  M.  Pittsley.  "Simulators 
for  Mariner  Training  and  Licensing: 
Functional  Specification  and  Training 
Program  Guidelines  for  a  Maritime  Cadet 
Simulator,  Draft  Technical  Report". 
National  Maritime  Research  Center,  Kings 
Point,  New  York,  1982. 
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computer  Aided  Instruction  and  Computer  Managed  Instruction  (CAI/CMI)  provide  train  n» 
systems  with  a  wide  assortment  of  capabilities.  These  capabilities  range  from  simply  record¬ 
ing  a  student  s  progress  to  eliminating  the  need  for  a  full  time  instructor.  This  paper 
explains  the  methodology  used  to  determine  the  amount  of  CAI/CMI  necessary  to  support  the 
needs  of  the  user.  It  also  discusses  the  management  of  the  interaction  and  coordination  of 
the  CAI/CMI  design  with  other  trainer  design  elements.  The  NTDS  Laboratory  Trainer  -  Device 
20F1 7  is  referred  to  as  a  quantitative  example  of  managing  CAI/CMI  development.  In  the 
development  of  Device  20F17,  the  courseware  developer’s  involvement  allowed  for  the  early 
definition  of  the  amount  of  CAI/CMI  needed  and  the  degree  of  automation  required.  This 
timely  input  in  the  proposal  phase  influenced  the  hardware  and  software  designM^Niccommo- 
daic  the  CAI/CMI  needs,  thus  minimizing  the  cost  of  including  these  features,  1 


INTRODUCTION 

As  the  sophistication  of  training  tasks  has 
increased,  the  necessity  for  greater  involvement 
ot  the  computer  as  an  assistant  to  the  instruc¬ 
tor  has  likewise  increased.  Since  many  tasks 
cannot  be  taught  solely  in  a  classroom  environ¬ 
ment,  computer  based  training  systems  have  proven 
they  can  significantly  improve  the  quality  of 
learn Lng . 

The  computer  has  also  been  used  to  promote 
the  capabilities  ol  the  laboratory  instructor. 
Many  ski  Lis  .are  difficult  to  explain  in  the  lab¬ 
oratory  by  an  instructor;  a  computer  based  sys¬ 
tem  can  demonstrate  these  for  him.  The  computer 
can  aLso  assist  the  laboratory  instructor  in 
evaluation  of  student  performance,  thus  improv¬ 
ing  his  productivity. 

With  the  rise  of  Computer  Aided  Instruction 
there  has  been  a  fragmented  approach  to  implemen¬ 
tation  methodology.  CM  has  been  implemented  in 
many  training  devices  over  the  past  few  years  to 
varying  degrees  and  with  varying  success.  In 
many  cases  the  implementations  have  not  met  the 
preconceived  goals.  In  other  cases  the  costs 
have  been  far  above  original  estimates.  However, 
in  some  situations  the  CA1  has  been  properly 
implemented  for  a  reasonable  cost. 

This  paper  will  attempt  to  define  a  method¬ 
ology  to  guide  the  CAL  implementation  decision. 

DEFINITION 

Computer  Aided  Instruction  (CAl)is  a  tool 
or  strategy  used  to  tench  cognitive  and  psycho- 
motor  competencies.  A  CA1  training  system 


offers  individualized  instruction  to  a  student 
without  having  an  instructor  present  at  all  times. 
CAT  is  sometimes  referred  to  as  Computer  Based 
Instruction  (CBT)  and  may  bo  divided  into  two 
categories:  Computer  Aided  Instruction  (CAT.)  and 

Computer  Managed  Instruction  (CMI), 

Capabilities  of  CAI 

CAI  includes  the  ability  to  generate  a 
learning  environment,  provide  instruction,  and 
allow  practice  sessions.  The  capabilities  de¬ 
rived  from  CAI  include  simulation,  stimulation, 
cues,  prompts,  and  feedback. 

For  the  purposes  of  this  paper  it  is  appro¬ 
priate  to  divide  CAI  into  intrinsic  and  extrinsic 
capabilities.  Intrinsic  capabilities  arc  those 
skills  that  a  trainee  would  receive  from  operat¬ 
ing  simulated  and/or  stimulated  equipment.  Com¬ 
puter  ^ s imulation  is  capable  of  generating  a 
learning  or  operational  environment.  Simulation 
is  considered  to  have  moderate  training  realism, 
is  cost  effective,  and  is  easily  implemented. 
Stimulation  uses  operational  equipment  which  pro¬ 
vides  a  higher  degree  of  training  realism.  The 
increased  fidelity  typically  requires  higher 
acquisition  cost  and  creates  a  more  difficult 
implementation.  Stimulation  has  on  additional 
advantage  in  that  it  uses  actual  equipment  with 
operational  software  and  can  easily  be  upgraded. 

Extrinsic  capabilities  add  to  a  learning 
environment  by  offering  knowledge  of  results.  It 
includes  cues,  prompts,  and  feedback.  Cues  and 
prompts  guide  the  student  toward  errorless  learn¬ 
ing  while  feedback  allows  the  student  to  learn 
from  his  mistakes.  Extrinsic  cues,  prompts,  and 
feedback  should  be  replaced  by  intrinsic  feedback 
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ns  proficiency  increases. 


Capabilities  of  CM1 

CMI  does  the  busy  work  for  the  instructor. 
It  allows  more  time  for  the  instructor  to  help 
the  student  develop  desired  techniques.  It  also 
has  the  potential  to  decrease  the  number  of  in¬ 
structors  required.  CMI  includes  computer  aided 
grading,  record  keeping,  and  exorcise  control. 


Another  disadvantage  of  CAI  is  the  lack  of 
flexibility  Of  the  curriculum.  When  designing 
a  training  system,  flexibility  must  be  considered 
An  instructor  can  modify  a  lesson  as  it  is  being 
presented  while  automated  systems  are  only  as 
flexible  as  their  design  allows.  More  flexibility 
means  additional  development  time  and  cost. 

SELECTION  METHODOLOGY 


Computer  aided  grading,  or  automated  per¬ 
formance  measurement,  is  a  CMI  function  that 
provides  a  means  to  grade  the  trainee  actions 
objectively.  Instruction  based  on  test  samples 
can  modify  the  learning  path  according  to  the 
specific  training  methods  employed.  For  example-,- 
three  such  training  methods  are  Linear,  Intrinsic- 
branching,  and  Mathetics.  The  Linear  Method  (1), 
developed  by  B.F.  Skinner,  provides  sequenced 
instruction  in  small  steps  where  the  student  is 
trained  to  respond  to  a  giv'a  stimulus.  The 
Intrinsic  branching  Method  ^ ^  ,  dove  loped  by  N.A. 
Crowder,  allows  for  student  mistakes  and  adjusts 
the  instruction  to  the  learner's  needs.  The 
Mathetics  Method  ( 3 )  j  dpve loped  by  T  Gilbert,  is 
a  methodical  approach  that  properly  arranges  key 
points  to  ensure  learning. 

Computer  aided  record  keeping  can  be  used 
to  track  students  through  an  entire  school  cur¬ 
riculum  or  just  for  an  individual  session.  A 
by-product  of  computer  aided  record  keeping  is 
the  establishment  of  trainee  historical  data. 


Before  selecting  the  computer  as  a  training 
tool,  the  task  itself  must  be  considered.  The 
task  to  be  analyzed  must  be  well  defined  and 
measurable.  Task  definition  is  done  in  a  task 
analysis  by  personnel  trained  in  analysis  pro¬ 
cedures. 

Task  Analysis 

A  task  analysis  identifies  the  jobs  that 
need  to  be  performed  in  the  operational  environ¬ 
ment.  It  also  defines  the  conditions  under  which 
the  task  must  be  performed  and  the  standards  that 
tell  when  the  task  is  mastered.  Some  tasks,  such 
as  perceptual,  volitional,  affective  and  judge¬ 
mental  tasks  are  not  observable  and  cannot  be 
graded  directly.  These  types  of  tasks  should  be 
taught  using  group  interaction.  Tasks  that  should 
be  considered  for  CAI/CMI  instruction  are  those 
requiring  cognitive  and  psychomotor  skills.  Each 
defined  task  must  include  main  intent  objectives 
or  indicators.  Indicators  that  can  be  graded 
must  be  developed  to  confirm  that  the  learner  has 
performed  an  unobservable  task. 


As  an  exercise  controller,  CMI  can  generate 
the  initial  environmental  conditions,  control 
each  exercise  event,  or  any  combination  in  be¬ 
tween.  The  computer  can  manage  an  entire  train¬ 
ing  session  where  cnc!'.  exercise  event  is  control¬ 
led  hv  the  CMI  system  or  be  used  to  only  estab¬ 
lish  initial  conditions. 

Limitations  and  Disadvantages  of  CA 1 

Although  CAI  is  effective  in  many  teaching 
situations,  several  inherent  limitations  and  dis¬ 
advantages  exist.  Consequently,  these  limita¬ 
tions  must  he  evaluated  when  considering  the 
incorporation  of  CAI  in  a  training  system.  CAI 
i. s  not  suited  to  teach  all  learning  competencies. 
It  is  not  well  suited  for  teaching  items  such  as 
volitional  (will),  affective  (emotions/ feelings) , 
interpersonal  and  judgemental  skills. 

Any  skill  that  requires  human  interaction 
to  develop  self-esteem  requires  a  sensitive 
learning  environment.  Volitional  and  affective 
competencies  present  a  real  challenge  to  a 
learning  system  due  to  their  interpersonal  re¬ 
quirements.  Teaching  judgmental  skills,  the 
ability  of  the  student  to  determine  alternatives, 
is  also  difficult  to  automate.  Each  task  must 
be  well  documented  and  the  objectives  must  be 
stated  in  measurable  behavioral  terms.  Docu¬ 
menting  these  tasks,  objectives,  and  standards 
require  analysis  which  normally  causes  develop¬ 
ment  cost  ami  schedule  to  be  prohibitive. 


Method  Means  Analys is 

After  completion  of  the  task  analysis,  a 
method  means  analy: is  should  be  performed.  A 
method  means  analysis  is  defined  as  a  listing  of 
possible  teaching  methods  available  to  the  devel¬ 
oper.  This  listing  includes  the  various  teaching 
methods  and  then  identifies  advantages  and  dis¬ 
advantages  inherent  in  each  method.  The  selection 
of  a  teaching  method  should  be  matched  to  the 
task  itself.  Questions  that  are  asked  during  a 
method  means  analysis  include: 

•  What  is  the  task? 

•  What  methods  are  realistic  to  teach  the 
task? 


•  What  are  the  advantages/d i sadvantages  of 
each  method? 

Method  selection  should  not  stop  at  one 
choice.  Alternative  methods  must  be  considered 
as  well  as  a  combination  of  methods.  Teaching 
using  CAI  is  one  available  method  to  consider. 

r 

The  determination  of  whether  a  combined 
approach  is  feasible  generally  becomes  apparent 
during  the  method  means  analysis  stage.  Examples 
of  some  general  strategy  selection  rules  arc: 

•  Review  alternate  methods  to  teach  a 
subj  ect . 
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•  Strive  to  use  a  combination  of  teaching 
strategies. 


•  Aim  to  ad!  novelty  to  the  teaching 
me t  hod . 

At  first,  the  computer  was  viewed  as  a  wav 
(o  re  institute  individualized  teaching.  The 
computer  was  introduced  as  a  panacea  to  curb 
training  costs,  increase  standardization,  de¬ 
crease  training  time,  and  increase  training  ef¬ 
fectiveness.  Unfortunately,  in  some  cases,  the 
computer  was  seen  as  a  replacement  for  the  teach¬ 
er.  Just  as  the  book  does  not  replace  the  teach¬ 
er,  the  computer  will  not  replace  the  teacher. 

'iiw  teacher’s  role  continues  to  change  and  the 
computer  can  be  used  to  supplement  and  comple¬ 
ment  Che  teacher. 


leaching  a  task  with  conventional  manual 
methods  is  the  alternative  to  automation.  An 
effective  combination  scheme  frequently  includes 
computers  and  conventional  teaching  methods.  A 
methods  means  analysis  will  indicate  this  type 
of  strategy  if  both  schemes  prove  appropriate  for 
teaching  the  desired  skill. 

In  all  cases,  the  computer  should  be  con¬ 
sidered  for  teaching  mundane  tasks,  presenting 
prearranged  situations,  and  leading  the  learner 
through  a  series  of  related  competencies  while 
the-  computer  grades  and  evaluates  the  progress. 
Interpersonal  skills,  tasks  that  are  not  well 
defined  or  tasks  without  a  compLete  task  listing, 
should  generally  be  considered  lor  teaching  by 
an  instructor. 

Tlie  Imp  lemon  t  a  t  ion  Decision 


One  output  of  the  task  analysis  and  method 
means  analysis  is  a  recommendation  to  include 
Cal  as  part  of  the  teaching  process.  While  the 
range  of  CAI  implementation  techniques  can  span 
the  complete  range  of  automated  system  capabil¬ 
ities,  a  reduced  group  of  training  system  unique 
items  has  boon  cataloged.  These  training  fea¬ 
tures  include: 

1.  Lxercise  Control  -  The  computer  provides 
pre-programmed  initial  scenario  conditions.  More 
extensive  implementations  can  include  computer- 
control  timed  events  and/or  allow  for  instructor 
intervention  at  any  time. 

2.  Prompts  -  Alphanumeric  cues  are  given 
to  a  student  to  direct  his  attention  towards 
specific  learning  objectives  within  a  scenario. 

3.  Instruction  -  Detailed  lessons  are 
given  on  the  computer  for  individualized  learn¬ 
ing. 

4.  Performance  Measurement  -  The  eva luat ion 
of  the  student's  progress  is  performed  by  computer 
controlled  and  evaluated  tests. 

5.  Feedback  -  Messages  or  alerts  are  pro¬ 
vided  to  inform  the  student  when  a  mistake  has 
been  made  or  a  task  has  been  performed  correctly, 


6.  Performance  Printout  ■  A  hard  copy  o«’ 
the  performance  measurement  results  for  a  single 
scenario. 


7.  Branching  -  The  pace  and  structure  of 
the  lesson  are  altered  based  on  specific  student 
responses.  Incorrect  responses  can  initiate  re¬ 
tries  or  specific  remedial  portions  of  the 
scenario. 


8.  Performance  Recording  -  Recording  of 
student's  records  and  grade  data  for  a  full  class. 
Maintenance  of  a  data  base  on  the  history  of  the 
classes  can  be  included. 


The  training  system  designer  should  review 
each  of  these  CAI  features  against  the  method 
means  analysis  to  determine  the  applicability  for 
the  selected  training  problem.  Each  training 
feature  should  be  reviewed  and  a  cost/benefit 
evaluation  made.  Figure  1  shows  a  flowchart  ap¬ 
proach  to  the  degree  of  CAI  implementation  method- 
ology.  The  capabilities  are  placed  in  a  hierar¬ 
chical  sequence  so  the  selection  process  stops 
once  a  no  response  occurs.  At  this  point  you 
have  defined  the  training  system  CAI  capabi 1 i t ies . 
For  example,  if  one  chooses  to  include  exercise 
control,  prompts,  instruction,  and  performance 
measurement  but  not  feedback,  the  CAI  capability 
of  the  system  is  then  defined.  No  other  features 
can  be  implemented  without  a  feedback  capability. 

After  determination  has  been  made  as  to  the 
extent  of  CAI  implementation,  an  overall  review 
is  in  order.  The  question  must  be  asked, 

"Should  CAI  be  implemented  even  if  it  can  be?" 

The  factors  needed  to  perform  this  analysis  fall 
into  two  categories:  performance  and  cost.  CAI 
must  be  justified  on  one  of  these  two  grounds. 

The  first  evaluation  factor  is  improvement 
m  teaching.  In  many  cases  CAI  will  improve  the 
quality  of  the  instruction  thereby  increasing  the 
proficiency  level  of  the  graduate.  As  a  result, 
educators  may  eliminate  or  decrease  the  amount 
of  time  spent  on  later  training.  In  other  cases, 
significant  improvement  in  the  effectiveness  of 
the  graduates  can  be  achieved.  Further,  CAI  may 
be  the  only  way  to  produce  a  course  graduate  that 
is  capable  of  performing  to  the  level  defined  in 
the  task  analysis.  In  other  cases,  CAI  may  im¬ 
prove  the  quality  of  the  course  to  allow  a  higher 
percentage  of  students  to  pass. 

The  second 'basis  for  consideration  of  CAI  is 
cost.  The  cost  here  is  the  ultimate  life  cycle 
cost  of  training.  The  addition  of  CAI  features 
may  significantly  reduce  the  quantity  of  instruc¬ 
tors  needed  to  teach  a  course. _ For  example,  in 

the  case  of  the  Electronic  Equipment.  Maintenance 
Trainer  (EEMT )  Device  1  IB  106  the  staff  for  the 
Electronics  Technician  (ET)  school  at  Great  Lakes 
had  a  student  to  instructor  ratio  of  6:1.  After 
adding  EEMT  the  ratio  increased  to  10:1.  As  a 
second  example,  in  Device  20F17,  the  NTDS  Labora¬ 
tory,  the  inclusion  of  these  features  allows  a 
student  to  instructor  ratio  of  4:1,  a  significant¬ 
ly  better  ratio  than  would  otherwise  be  possible 
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•  EXERCISE 
CONTROl 


*  EXERCISE 
CONTROl 

•  PROMPTS 


As  was  seated  above,  the  inclusion  of  CA1 
can  not  only  improve  the  quality  of  the  graduate 
ya:  nun  in,  i  ze  the  need  for  follow-up  training. 

.  U,‘,turi'«  -'ll so  be  used  to  minimize  the 

orientation  time  of  instructors  with  the  Training 
evi.ee.  A  menu-driven  system  written  in  the 
oner's  language  simplifies  operation.  This  may 
allow  tor  decreased  device  specific  instructor 
t  ra ining . 


When  to _ Imp  1 ement  CA1 

il  an  analysis  of  the  particular  learning 
Situation  has  indicated  that  CAI  should  be  incor¬ 
porated  as  part  of  the  training  device,  the  next 
logical  step  is  to  ascertain  when  the  CAI  should 
be  incorporated .  This  decision  can  be  reflected 
in  the  specification,  which  should  define  the 
amount  of  CM  required. 


with  supervision.  The  device  consists  of  04 
operator  trainee  stations  grouped  into  sixteen 
stand-alone  suites.  Each  suite  contains  four 
simulated  OJ-194(V)/UYA-4(V)  look-alike  PPJ  con¬ 
soles,  four  simulated  AN/UPA-59( V)2  look-alike 
IFF  decoders ,  and  a  mini-computer  with  associated 
peripherals  and  interfaces.  One  of  the  1.0  suite* 
is  classified  as  the  master,  which  is  capable  of 
scenario  generation.  The  configuration  of  a 
typical  student  suite  is  shown  in  Figure  3.  The 
device  contract  also  requires  computer  operation-, 
al  software  to  control  the  trainer,  60  hours  of 
preprogrammed  scenarios  with  associated  course¬ 
ware,  and  all  classroom  and  laboratory  curriculum. 
The  trainer  software  also  emulates  a  DDC-37  class 
NTDS  operational  program. 

Input/User  Mode 


However,  the  supplier  of  the  training  device 
controls  the  management  strategy  for  CAI.  This 
strategy  includes  deciding  the  timing  of  the  in¬ 
volvement  of  the  courseware  developers.  In  cases 
uln  ie  the  generation  oi  training  scenarios  is 
required  early  in  the  development  process,  the 
timing  is  significant.  In  other  cases  the  course¬ 
ware  developers  might:  not  be  needed  until  the 
device  design  has  proceeded  past  t lie  critical 
design  review  stage. 

Uiere  are,  however,  significant  risks  in 
allowing  the  design  architecture  to  be  frozen 
without  a  full  evaluation  of  the  effects  of  CAI 
on  the  trainer.  In  addition,  tin.*  courseware 
developers  can  provide  a  void  representing  the 
umei  within  the  contractor's  project  organizat  ion. 

While  it  may  he  tempting  to  delay  the  use  of 
the  courseware  developers  to  save  cost,  there  is 
.i  penalty  tor  this  approach.  When  the  course¬ 
ware  developers  are  early  participants  in  the 
piojeci  team,  they  can  make  significant  observa¬ 
tions  regarding  dot  iciencies  in  the  trainer  ar¬ 
chitecture.  Any  gaps  may  prohibit  the  efficient 
anti  effective  implementation  of  the  CM  required 
by  the  specification.  The  involvement  of  the 
courseware  developers  during  the  system  design 
phase  can  improve  the  quality  of  the  overall 
training  device  design  and  reduce  the  total  im¬ 
plementation  cost. 


The  0J- 194(V)  PP1  look-alike  console  is  cap¬ 
able  of  responding  to  input/user  operations  which 
deal  primarily  with  evaluation  of  NTDS  data  to 
supplement  the  tactical  decision  making  process. 
User  functions  are  performed  by  such  operators  as 
Force  Weapons  Coordinator,  Ship  Weapons  Coordina¬ 
tor  and  Air  Intercept  Controllers.  Input  functions 
Include  air  detector  tracker,  surface/subsurface 
tracker,  identification  operator  and  track  super¬ 
visor.  A  typical  PPl  display  seen  by  these  oper¬ 
ators  is  shown  in  Figure  4. 

Tactical  Scenarios 

Preprogrammed  computer  generated  scenarios 
have  been  developed  to  portray  specific  tactical 
situations  within  the  confines  of  the  tactical 
environment.  The  scenarios  emphasize  input  oper¬ 
ator  responses  and  are  structured  to  provide 
fi t teen  separate  levels  of  achievement.  Scenarios 
permit  instructor  intervention  to  freeze  and  re¬ 
play  at  any  5  minute  operating  point  during  in¬ 
struction.  The  instructor  is  also  able  to  select 
scenario  speed  from  a  range  of  IX,  2X,  and  4X 
no i null  operating  speed.  The  device  is  designed 
to  enable  operation  of  a  common  scenario  within 
a  four  console  suite.  Each  scenario  independent  ly 
responds  in  real-time  to  the  respective  operator 
console  interactions.  The  PPl  display  provides 
realistic  video  displays  of  targets  and  media  as 
well  as  full  NTDS  symboloby. 


DEVICE  2 0 F 1 7  EXAMPLE 

An  example  of  how  CAT /CM I  was  selected  and 
implemented  is  the  Device  201*17  (NTDS  Laboratory). 
It  was  designed  as  a  generic  basic  operator 
ski  11s  simulator  using  Levels  of  achievement  to 
teach  simple  to  complex  tasks  in  a  sequential 
order.  Figure  2  is  an  artist  's  concept  of  Dev ico 
20F17. 


Target  Dynamics 


The  device  has  the  capability  to  maintain 
track  data,  provide  motion  simulation,  and  dis¬ 
play  in  real-time  up  to  24  active  exercise 
vehicles  (targets)  simultaneously  per  console. 
Each  of  these  targets  is  pre-nrogrammed  as  a 
specific  type  of  air,  surface,  or  sub-surface, 
target  with  defined  course,  speed,  altitude  and 
maneuvering  parameters. 


System  Dv.sc  r  ipt  ion 

The  NTDS  Laboratory  was  designed  to  simulate 
the  OJ  -  1 94  -  ( V )  /  IJ  YA  -4  ( V  )  PPL  Data  Display  Conso-e 
in  all  respects.  Tin;  device  provides  dynamic 
training  in  the  visual  and  tactile  responses  re¬ 
quired  to  achieve  the  NTDS  console  manipulative 
and  interpretive  skills  necessary  to  stand  a 
Condition  III  (Normal  Peacetime  Steaming)  watch 


Target  Behavior 

Preprogrammed  air  launch  platforms  are  sim¬ 
ulated  with  initial  course,  speed,  and  altitude, 
and  each  has  unlimited  time-designated  inflection 
points  where  preselected  course/speed  changes 
automatically  occur.  Air,  surface,  or  subsurface 
launched  antiship  cruise  missiles  (ASCM)  follow 
altitude  flight  profiles  similar  to  those  of 


NTDS  LABORATORY 
TRAINER-DEVICE  20F17 


Figure  2  Artist  Sketch 
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actual  plat  tonus .  Tlu*  capability  exists  at  the 
student  consoles  and  in  scenarios  to  assign 
Identification  Friend  or  Foe  (IFF)  codes  to  each 
ait  target.  Each  target  has  the  capability  of 
being  designated  as  an  active  jammer  by  the 
scenarios.  The  number  of  active  jammers  in  any¬ 
one  scenario  is  limited  to  ten. 

V  id i-o  Display 

liie  device  simulates  weather  clutter,  sea 
return,  landmass,  barrage  jamming  effects,  chaff, 
II- f  video  and  symbology.  Each  is  implemented  in 
response  to  control  commands  from  the  preprogram¬ 
ed  scenarios  for  I* PI  display  on  the  look-alike 
c  on. so  1 e . 

CAi  IMPLEMENTATION'  FOR  DEVICE  20F17 

Iho  Device  2UFi7  was  designed  as  a  generic 
basic  operator  skills  simulator  to  avoid  teach¬ 
ing  the  ever  changing  software  and  the  numerous 
Consoles  in  use  in  the  fleet  today.  This  is 
possible  because  the  basic  jobs  of  the  input 
operators  have  not  changed  in  twenty  years.  Pro¬ 
grams  and  hardware  have  changed  the  method  of 
data  cut  rv/ext  fact  ion  and  in  some  cases  the  Lo¬ 
cation  i'f  the  controls,  but  the  jobs  have  remain¬ 
ed  intact.  Device  2 OF  1 7  teaches  these  basic 
skills  using  one  console  and  a  generic  NTDS  soft¬ 
ware  program. 

Once  the  skills  are  mastered,  the  students 
are  able  to  apply-  Lists  knowledge  on  different 
consoles,  using  d i t ! even l  NTDS  programs.  Generic 
t  r  a  i  n  i  ng  a  1.  1  ows  basic  skills  t  rn  i n  ing  to  be 
insulated  from  the  impact  of  the  continuing 
changes  in  operational  hardware. 

Task  An a  I y  s i s 

The  Device  20F17  development  started  in 
11)77  with  a  Needs  Ar  lysis  *>‘*)  which  was  verified 
in  1980  at  the  beginning  of  the  present  develop¬ 
ment.  The  needs  analysis  defined  the  input 
operator's  basic  skills.  Subject  matter  experts 
that  were  trained  in  instructional  systems deve 1- 
opment  were  able  to  work  with  the  NTDS  operators 
at  F L KCOM BAT RAC EN LANT  (Dam  Neck,  VA)  to  define 
the  job  tasks  without  a  formal  task  analy-sis. 

The  task  listing  was  limited  to  the  performance 
itself;  the  conditions  and  standards  were  not 
Je lined.  The  tasks;  were  sequenced  and  grouped 
into  levels  of  achievement.  Levels  of  achieve¬ 
ment  was  a  term  coined  to  place  a  more  positive 
approach  co  learning  than  the  phrase  levels  of 
difficulty.  The  task  analysis  also  did  not 
define  quantitative  values  for  detecting,  enter¬ 
ing  and  tracking  sk'Lls.  No  objective  data  on 
these  performance  standards  was  available  from 
the  fleet. 

Method  Meatus  Atm  1  vs  is 

The  same  1977  analysis  study-  compared  a  1  ter- 
native  methods  of  teaching  the  NTDS  operator 
tasks.  Advantages  and  disadvantages  of  each  were 
listed,  including  cost.  Recommendations  for 
training  system  selection  were  made  as  a  part  of 
the  study.  Each  ill  ti-rnat  ivo  was  compared  to  use 
of  the  operatiou.il  equipment  iti  the  trainer  and 


the  following  alternatives  were  considered: 

1.  Look-alike  consoles  with  sixteen  com¬ 
puters.  This  is  the  approach  that  was  adopted 
as  the  Device  20F17. 

2.  Look-alike  consoles  with  one  computer. 
This  approach  tied  all  of  the  consoles  together; 
however,  no  provisions  were  made  for  degraded 
mode  when  the  one  computer  failed, 

3.  A  slide-tape  training  system.  The  cost 
was  fairly  low;  however,  the  tactile  "feel"  and 
capability  for  future  expansion  was  limited. 

4.  Operational  equipment  driven  by  a  sim¬ 
ulator  was  considered.  The  fidelity  was  good; 
however,  the  cost  was  excessive. 

CAI  Features 

Since  the  quantity  of  students  requiring 
training  in  NTDS  basic  operator  skills  exceeds 
2000  per  year,  it  was  determined  that  reduction 
in  the  instructor's  workload  was  important;  an 
instructor  to  student  ratio  of  1:4  was  dictated. 
This  requirement  then  led  to  cons iderat i „n  of 
various  CAI  features  to  aid  the  instructor  and 
provide  supplementary  tutoring  to  the  students. 

A  comparison  of  the  flowchart  approach  of 
Figure  1  to  the  needs  of  Device  20F17  demonstrates 
the  process  of  consideration  of  CAI  features  for 
a  trainer.  In  order  to  reduce  the  instructor's 
workload  during  laboratory  sessions,  it  was 
decided  to  use  preprogramed  scenarios  (EXERCISE 
CONTROL!.  Since  one  instructor  cannot  be  con¬ 
stantly  in  contact  with  all  four  students,  com¬ 
puter  generated  messages  on  an  instructional/ 
tactical  CRT  (PROMPTS)  synchronized  to  the 
scenario  was  provided.  The  student's  learning 
process  was  also  enhanced  by-  the  inclusion  of 
capabilities  for  computer  display  of  detailed 
step-by-step  procedures  (INSTRUCTION)  synchronized 
to  the  scenario.  Since  many  of  the  skills  such 
as  ball  tab  positioning-accuracy  can  be  quant  i  f  ied  , 
computer  generated  tests  and  evaluation  of  student 
behavior  (PERFORMANCE  MEASUREMENT)  was  included. 
The  timing  of  these  tests  was  also  controlled  by 
the  scenario.  It  was  alro  concluded  that  real¬ 
time  results  of  these  tests  (FEEDBACK)  should  be 
available  to  the  student  in  the  early  phase  of 
training.  The  scenario  author  would  define 
whether  the  test  results  were  to  be  displayed  to 
the  student  and/or  stored  for  printout  at  the 
end  of  the  lesson  (PERFORMANCE  PRINTOUT). 

Other  CAI  features  were  then  considered. 
Real-time  modification  of  the  lesson  based  on 
student, proficiency  (BRANCHING)  was  rejected  as 
being  more  sophisticated  than  was  desired  for  the 
trainer.  The  high  student  load,  necessitating  a 
lock-step  curriculum  was  partially  in  conflict 
with  BRANCHING,  where  the  lesson  time  can  vary 
from  student  to  student.  The  additional  implemen¬ 
tation  cost,  courseware  development  time,  and 
program  risk  were  also  factors  in  the  decision. 

The  establishment  of  a  historical  data  base 
( PERFORMANCE  RECORDING)  was  considered,  but  view¬ 
ed  as  beyond  the  scope  of  the  training  require¬ 
ments! 


° *•  the  Courseware  Developers 

!  courseware  developers ,  if  not  subject 
m.'it-ter  experts,  had  to  learn  the  tasks  being 
tduyht  so  they  could  synthesize  the  material 
lh°y  WlMV  1,1  c  iintite  with  the  user's  language, 
equipment,  and  job.  Courseware  developers  that 
were  Knowledgeable  of  equipment  operations  and 
performance  measurement  requirements  assisted  in 
tne  hardware  and  software  development. 

In  the  development  of  Device  20F17,  course¬ 
ware  developers  served  a  valuable  role  in  repre¬ 
senting,  within  tne  project  team,  the  needs  of 
the  user.  since  the  motivation  of  the  course¬ 
ware  dove lopei  paralleled  those  of  the  user,  the 
influences  provided  by  the  courseware  personnel 
allowed  the  project  design  to  better  reflect  the 
user's  desires.  Inis  participation  in  the  design 
concept  incorporated  subtleties  in  the  training 
device  which  maximized  its  training  effect- ive- 
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Courseware  developers  on  Device  20F17 
assisted  in  Jefining  the  hardware  requirements 
needed  for  performance  measurement.  All  hard¬ 
ware  control;  and  indicators  are  not  needed  to 
operate  as  the  actual  equipment  functions.  By 
defining  which  controls  and  indicators  are  need¬ 
ed  lor  the  degree  of  authenticity  required,  the 
hardware  design  and  construction  was  simplified 
with  a  corresponding  cost  savings. 

lh,e  Device  20F17  hardware  design  architec¬ 
ture  allowed  for  many  variations  of  hardware 
design..  One  design  approach  considered  used  an 
intelligent  look-alike  console  with  significant 
processing  power  located  at  each  console.  While 
th. is  approach  is  acceptable  from  an  implementa¬ 
tion  perspective,  it  complicates  the  task  of 
analyzing  all  student  inputs  and  providing 
quantitative  performance  evaluation.  The  CAT 
requirement  which  was  included  in  the  contract 
specification  directed  the  hardware  design  to¬ 
wards  a  dumb  look-alike  console  with  all  student 
switch  and  control  actions  available  to  the  host 
simulation  computer.  This  hardware  design 
decision  simplified  the  incorporation  of  CAI 
throughout  the  trainer.  It  provided  operator 
inputs  and  responses  to  the  software,  thereby 
simplifying  the  task  of  implementing  the  CAI 
so  f  tware . 

Courseware  Developers  Influence  on  So f twa r e 

The  CAI  unique  software  requirements  for 
Device  20F17  included  an  offline  scenario  gener¬ 
ation  program  to  allow  scenario  authors,  with 
little  computer  background,  to  generate  scenarios . 
In  addition,  real  time  programs  implemented 
specifically  for  CAI  provide: 

•  Prompts  to  be  displayed  for  the  student. 

•  On-line  feedback  messages  to  the  student, 

•  Real  time  performance  measurement. 


This  substantial  software  effort  was  signif¬ 
icantly  impacted  by  the  courseware  developers 
Discussions  with  the  software  project  engineer 
and  the  program  manager  resulted  in  modifications 
to  the  original  design  concepts.  These  modifica¬ 
tions  made  the  programs  more  usable  to  the  course¬ 
ware  developers  and  thus  simplified  their  job. 
Those  Changes  also  simplify  the  user's  job  to 
modify  the  delivered  courseware  or  generate  new 
scenarios  as  required. 

The  most  striking  changes  occurred  in  the 
scenario  generation  program.  With  the  influence 
of  courseware  people,  the  scenario  development 
program  evolved  into  a  user  oriented  authoring 
system.  The  user  is  led  through  the  program  and 
prompted  for  appropriate  entries  by  a  sophisti¬ 
cated  generation  and  editing  capability.  The 
program  also  includes  scenario  verification  and 
error  checking  to  insure  that  preprogramed  tar¬ 
gets  are  maintained  within  a  realistic  environ¬ 
ment;  i.e.  surface  ships  do  not  cross  a  land 
edge  barrier. 

The  performance  measurement  program  was 
also  significantly  enhanced  and  provides  sophis¬ 
ticated  and  meaningful  performance  measurement 
for  a  wide  range  of  significant  trainee  measure¬ 
ment  situations. 

SUMMARY 

Computer  Aided  Instruction  has  been  firmly 
established  as  a  requirement  in  many  existing 
and  future  training  devices.  It  can  signi  f  icslitly 
improve  the  training  capability  when  properly 
implemented. 

This  paper  has  presented  a  simplified  ap¬ 
proach  which  can  assist  in  making  critical  CAI 
implementation  decisions.  The  examples  of  CAI 
features  on  Device  20F17  provide  lessons  which 
can  be  applied  to  other  training  applications. 

With  the  proper  approach  CAI*  can  enhance  almost 
.any  trainer. 
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ABSTRACT 


‘Millions  of  government  dollars  are  being  spent  or  documentation  of  the  computer 
program  systems  of  Aircrew  Traininq  Devices  every  year  under  the  premise  of  i fe  cycle 
support.  As  the  costs  of  Aircrew  Training  Devices  increase,  it  bet .me.;  impe rati ve  that  a 
complete  evaluation  be  done  to  decrease  documentation  costs  This  pape*'  discusses  the 
following  components  of  the  documentation  issue. 

(a)  HJhat  documentation  is  necessary  for  development  and  acquisition  tracking? 

Do  we  use  it? 


support?, 


(b)  -What  documentation  is  necessary  for  three  inaior  manning  levels  during 

? 

\  .  V 

(c)  "How  much  is  "blue  suitv  maintenance  costing  as  part  of  ATD? 

(d)  -Commercial  Practices  Documentation  -  is  it  any  gooa? 

(e)  -Will  computerization  save  us? 


(f)  *Where  can  dollars  be  shaved? 

INTRODUCTION  \ 

Since  late  in  1973,  the  Air  Force  has  been 
reemphasizing  overall  aircrew  training  to  include 
a  higher  dependence  upon  Aircrew  Training  Devices 
(ATDs)  for  accomplishment  of  simulator  unique 
training.  Inherent  to  this  change  in  emphasis  is 
the  increase  in  total  dollars  expended  on  ATD 
acquisition  and  follow-on  support.  As  the 
economy  weakens,  the  cost  of  people  intense  tasks 
continues  to  increase.  Two  of  the  tasks  which 
most  effect  ATDs  are  he  writing  of  software 
programs  and  subsequent  documentation .  The  ATDs 
currently  procured  are  more  complex  and  software- 
intensive  than  ever  before.  With  this  is  the 
overall  increase  in  ATDs  cost,  and  a  drive  to 
decrease  wherever  possible  the  high  cost  contri¬ 
butors,  so  that  more  devices  can  be  acquired 
without  degradation  of  overall  training  effective¬ 
ness  This  paper  defines  what  documentation  is 
needed  for  Computer  Program  System  (CPS)  Develop¬ 
ment,  acquisition  tracking,  and  the  different  ATD 
support  philosophies.  The  cost  of  organic  main¬ 
tenance  is  evaluated.  In  addition,  commercial 
documentation  and  trends  in  computerization  of 
CPS  documentation  are  also  discussed.  Finally, 
three  recommendations  are  made  which  would  help 
decrease  the  total  cost  of  CPS  documentation  in 
ATDs. 

DEVELOPMENT  DOCUMENTATION 
What  is  Needed 

The  life  cycle  of  an  ATD  can  be  divided  into 
two  major  phases,  acquisition  and  support.  Each 
phase  has  '•istinct  and  different  requirements  for 
documentation  of  the  CPS.  During  the  acquisition 
phase,  where  the  CPS  is  designed  and  developed, 
three  tasks  are  aided  by  CPS  documentation: 

(1)  Document  the  development  of  the  CPS 
and  its  design; 


(2)  Program  reporting,  status,  and  track¬ 
ing  to  the  acquisition  agency;  and 

(3)  Support  testing  of  the  device. 

These  three  tasks  generate  entirely  different 
requirements  for  documentation  and  each  will  be 
covered  independently. 

Design  Documentation 

The  CPS  Design  Documentation  is  produced  to 
show  how  the  CPS  is  developed  and  what  the  final 
design  will  be.  At  the  Preliminary  Design  Review 
(PDR)  three  areas  of  the  design  will  be  documented. 
First,  the  allocation  of  each  system  into  hardware 
functions  and  software  functions  must  be  identified 
and  ftozen  as  much  as  possible.  Second,  any 
previously  used  model  which  has  been  selected  for 
use  on  the  ATD  must  be  identified  and  described. 
Finally,  ai. .  most  importantly,  .  \  overview  document 
must  be  provided  that  defines  how  the  system  will 
work . 

At  each  successive  design  review  the  design 
document  should  become  more  definative  until  the 
Critical  Design  Review  ( CDR )  when  the  final  version 
should  be  complete.  After  the  CDR,  since  some 
changes  are  inevitable,  change  pages  should  be 
issued  to  support  the  corresponding  design  change. 
The  type  of  documentation  required  is  dependent  on 
the  type  of  functional  system.  ATD  real-time 
applications  consist  of  two  types  of  systems: 
one  based  on  a  mathematical  model  of  the  aircraft, 
and  one  which  is  somewhat  task  oriented. 

Systems  based  on  a  mathematical  model  have 
special  documentation  requirements,  in  both  hard¬ 
ware  and  software,  to  show  design  correlation 
between  the  real-world  system  and  the  simulated 
system.  To  document  this  most  explicitly  would 
require  two  functional  block  diagrams,  equivalent 
to  figure  1,  and  supporting  narrative  descriptions 
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FIGUSr  1  FUNCTIONAL  BLOCK  DIAGRAM 


of  both  the  real  and  simulated  systems.  This 
would  provide  a  top  level  understanding  of  the 
design  approach,  per  function,  for  both  contrac¬ 
tor  and  government  engineers  to  follow.  A 
detailed  presentation  of  the  simulator  system’s 
hardware/software  interfaces  should  be  defined. 
Probably  the  most  important  interface  would  be 
with  a  user's  guide,  and  include  complete 
operating  procedures.  In  order  for  a  task 
oriented  system  to  be  defined  completely  all 
outside  interfaces  must  first  be  established  and 
discussed  during  early  design  reviews.  One 
obvious  and  important  example  would  be  the 
instructional  aids/controls. 

Status  Docnme nta u ion 

Documentation  necessary  to  assist  the  acqui¬ 
sition  agency  in  tracking  the  development  effort 
is  the  second  type  of  development  documentation 
required.  The  System  Engineering  Management 
Han,  Computer  Program  Development  Plan  and 
Firmware  Development  Plan  all  provide  development 
planning  information  and  schedules  to  both  con¬ 
tractor  and  government  personnel.  Other  related 
plans  are  for  Configuration  Management  and 
Quality  Assurance.  When  defined  completely  and 
used  as  a  rigid  model  against,  which  develop¬ 
ment  activities  are  based,  these  plans  are  very 
useful  . 

Outside  of  the  planning  document,  additional 
data  should  be  generated  for  tracking  the  f.PS. 

The  most  widely  used  is  a  representation  of  CPS 
allocation  in  a  tree  form.  This  provides  a  guide 


to  how  the  CPS  is  designed,  a  base  for  implemen¬ 
tation  of  planning  documents  and  size/time  track¬ 
ing.  Size/time  track!,  g  permits  frequent  moni¬ 
toring  of  critical  computer  resources. 

Test.  Documentation 


Very  little  additional  documentation  is 
necessary  to  support  the  ATD  testing,  as  long  as 
the  initial  design  documentation  is  correct.  An 
alphabetic  cross-ref erpna?  of  engineering  terms 
ann  program  mnemonics,  units  and  modules  that 
use/i hange  these  variables,  is  used  the  most 
frequency,  a  I  on  q  with  other  printouts,  such  as 
program  listings,  memory  allocation  listings,  and 
data  pool  lists.  The  most  important  issue  for 
test  documentation  is  that  it  be  correct,  avail¬ 
able,  and  maintained  in  a  current  state. 

Do  We  Use  Development  Document  a  Hon  ? 

All  documentation  produced  during  develop¬ 
ment  is  utilized  to  some  degree.  If  design 
documentation  is  provided  before  the  design 
reviews  to  enable  a  pre-design  review  evaluation, 
then  the  design  reviews  become  more  detailed  arid 
comprehensive.  This  increases  government  aware¬ 
ness  of  the  design  and  minimizes  surprises  as 
design  changes  develop  and  testing  begins. 

Status  documentation  keeps  the  government  aware 
of  the  contractor's  progress,  even  if  it  is 
toward  a  schedule  slip.  It  is  far  more  cost 
effective  to  catch  such  a  slip  one  month  after 
CPR  than  twu  months  before  test  .  Good  test 
documentation  assures  a  more  complete  test 
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of  the  device,  for  both  the  contractor  and  the 
USA  r . 

Development  documentation  is  also  used  as  the 
basis  for  support  documentation.  Fven  commercial 
documentation  is  based  upon  the  definition  of  the 
CPS  product  which  is  internally  produced  during 
the  development  cycle.  Additional  data  items 
required  and  specified  formats  make  up  all  the 
major  differences  in  support  documentation. 

SUPPORT  DOCUMENTATION 
Maintenan ce  Concepts 

Normal  definitions  of  maintenance  appropriate 
to  hardware  are  not  easily  adapted  to  software 
maintenance.  We  can,  however,  divide  software 
maintenance  approaches,  including  modifications, 
correction  of  latent  deficiencies  and  enhance¬ 
ments,  between  organic  (blue  suit),  hybrid  and 
full  contractor  maintenance  concepts. 

In  the  blue  suit  concept,  mil i  ar  ATD  main¬ 
tenance  technicians  handle  software  ..  'port. 

Little  training,  tour  length  and  retention  are  the 
primary  contributors  toward  lack  of  effectiveness. 

A  hybrid  is  a  compromise  between  the  total 
blue  suit  maintenance  and  :  'Tal  contractor  main¬ 
tenance.  It  is  based  upor  uae  augmentation  of 
the  blue  suit  concept  with  engineering  educated 
personnel,  either  government  or  contractor.  With 
this  concept,  engineering  capability  is  increased 
and  long-term  civilian  augmentation  can  resolve 
problems  created  by  military  duty  tours  and  re¬ 
ass  i  gnnients . 

The  third  level  or  type  of  software  mainte¬ 
nance  is  contractor  support  or  Contracted 
Logistics  Support  (CLS).  Contractor  support 
provides  an  increase  in  engineering  skilled 
personnel  and  consistency  in  support.  Usually, 
a  support  team  will  be  staffed  by  one  or  two 
qualified  field  engineers  and  augmented  by  newly 
graduated  engineers,  with  very  little  ATD  experi¬ 
ence.  This  concept  does  provide  for  an  experience 
base  to  be  built,  unless  the  service  contract  is 
recompeted  within  two  years  and  awarded  to  a 
second  company.  Lven  then,  it  is  not  uncommon 
for  incumbent  personnel  to  transfer  employment  to 
the  second  company.  This  concept  has  the  added 
advantage  of  making  the  originator  of  the  support 
documentation  directly  responsible  for  its  adequa¬ 
cy  and  providing  a  tie  to  the  originator  for 
additional  support  when  needed. 

Support  Docume n ta  t  i  on 

Fach  of  the  three  concepts  of  software  main¬ 
tenance  has  different  support  documentation  re¬ 
quirements.  Due  to  limited  training  and  retention, 
the  total  blue  suit  concept  has  the  greatest  need 
for  quality  documentation  and,  therefore,  has  the 
higher  associated  cost.  The  maintenance  tech¬ 
nicians  need  to  use  the  documentation  both  top- 
down  and  bottom-up.  Ihe  top-down  study  approach 
is  used  to  teach  new  technicians  about  simulation 
in  general,  and  experienced  technicians  how  the 
specific  ATD  system  works.  Tor  this  study 
approach  extra  documentation  is  required  to  pro¬ 
vide  simulation  technology  and  development  phi¬ 
losophy,  as  well  as  detailed  step-by-step 


procedures  defining  how  to  use  the  available  soft- 
warp  A  hoitnin-un  approach  is  used  when  a  latent 
deficiency  or  propused  enhancement  is  identified, 
to  track  back  from  listings  to  what  math  equations/ 
models  are  applicable.  For  this  study  approach 
extra  documentation  is  required  *o  provide  this 
backwards  tracking  since  development  documentation 
is  evolved  going  in  the  opposite  direction. 
Modifications  of  any  size  are  usually  contracted 
out . 

With  the  addition  of  engineering  personnel, 
the  bottom-up  approach  is  usually  eliminated.  An 
understanding  of  system  and  basic  engineering 
concepts  available  for  a  long  term  shou’d  replace 
this  approach.  Small  modifications  should  be 
supportable  with  this  augmented  blue  suit  concept, 
even  though  many  of  the  larger  modifications 
would  still  be  contracted  out. 

The  third  level  of  software  support  should 
handle  all  but  major  modifications  through  the 
contracted  support  team.  Additional  documentation 
for  simulation  technology  and  development  phi¬ 
losophy  would  not  be  required  since  the  two  field 
engineers  would  be  able  to  instruct  other  team 
members.  Also,  it  the  field  engineers  had  pre¬ 
viously  participated  as  part  of  the  on-site  test 
team,  only  top  level  procedures  for  software 
usaqe  would  be  required.  Actually,  each  contract 
<iv.  rd  should  include  an  overlap  of  contractors 
to  ensure  no  dpnr.idation  of  support  during  t Fie 
transition  period  from  acceptance  testing  to 
operational  training.  The  documentation  required 
for  CLS  is  equal  in  most  cases  to  development 
documentation. 

BLUE  SUIT  MAINTENANCE  COSTS 
R  eenl i stments 

One  of  the  greatest  differences  between  the 
USAF  maintenance  teams  and  commercial  simulator 
companies'  support  teams  is  the  average  experience 
level.  This  is  caused  by  a  decreasing  reenlist¬ 
ment  statistics  trend.  In  1981  only  twenty- five 
percent  of  all  first  term  ATD  maintenance  tech¬ 
nicians  ( AFSC  341 XX)  elected  to  reenlist.  This 
can  be  compared  to  a  forty-two  percent  average 
for  all  Air  Torce  specialties.  This  1981  first 
term  reenlistment  also  compares  poorly  with  1979, 
when  thirty-seven  percent  of  ATD  maintenance 
technicians  reenlisted.  Of  all  second  term  ATD 
maintenance  technicians,  only  forty-three  percent 
reenlisted,  compared  to  seventy-one  percent  for 
all  specialties.  (1) 

Ira  ini ng 

As  each  newly  enlisted  service  member  attains 
an  ATSC  rating  he  is  sent  to  specialized  tech¬ 
nical  training  to  master  the  skills  required  for 
full  qualification.  To  train  an  enlisted  member 
in  the  ATD  maintenance  fields  (all  AFSC  341XXT, 
it  will  cost  an  average  of  $  1  ? . 4 7 7  in  198* 
dollars,  compared  to  $8,799  in  1980  dollars  (see 
Table  1).  (?)  Taese  costs  include  incentives 

pay,  selective  reenlistment  bonuses  and  profi¬ 
ciency  pay.  They  are  based  on  t Fie  cost  per 
guaduate  for  training  required  at  a  basic  skill 
level,  acquisition  costs,  basic  training  costs, 
pay  and  allowances  while  in  pretechnical  training, 
permanent  change  of  station  after  training,  and 
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TABLE  1  ENLISTED  PERSONNEL  ACQUISITION  COST/GRADUATE 


AFSC 

EY80  S 

FY82  $ 

34131 

Instrument  Training  Specialist 

3,675 

4,513 

34132 

Defensive  Systems  Training  Specialist 

6,966 

8,894 

34133 

Analog  Flight  Simulator  Specialist 

10,092 

14,567 

34134 

Digital  Flight  Simulator  Specialist 

10,161 

14,859 

34135 

Analog  Nav  Simulator  Specialist 

10,434 

15,212 

34136 

Digital  Nav  Simulator  Specialist 

11,450 

16,819 

pay  and  allowances  for  leave  accrued  during 
training.  They  do  not  include  training  to  attain 
an  AFSC  rating,  on-the-job  training,  nor  Type  I 
training  after  reporting  at  the  new  duty  station. 

Government  Documentation  Costs 

Based  on  cost  data  and  educated  estimates  it 
costs  three  to  five  times  more  to  buy  tne 
additional  software  documentation  necessary  for 
blue  buit  maintenance  than  commercial  data.  The 
variances  can  be  explained  by  differences  in 
commercial  documentation  provided  by  different 
companies.  The  percentages  add  up  to  large  sums 
cf  money,  over  five  million  dollars  on  one  pro¬ 
gram  just  for  the  basic  documentation  costs. 

If  we  add  in  the  cost  of  training  the  seventy- 
five  percent  of  first  term  ATD  maintenance  tech¬ 
nicians  that  did  not  reenlist,  an  alternative 
solution  becomes  attractive. 

COMMERCIAL  DOCUMENTATION 

standards 

There  are  no  standards  equivalent  to  those  in 
the  military  co  define  what  commercial  documen¬ 
tation  v'ans.Jfs  of,  except  some  standardi zation 
accomplished  t.UiOiigh  the  computer  industry 
(ANTI!,  etc.!.  I  ach  company  publishes  inH  ••ndual 
standards  and  defines  how  th?ir  company  will 
document  computer  programs.  This  is  indirectly 
influenced  by  military  standards  for  companies 
with  a  large  share  of  the  government  market, 
although  such  compaf abi 1 i ty  of  standards  is 
strictly  voluntary.  When  a  company  oublishes 
documentation  standards  it  is  up  to  that  company 
to  enforce  those  stindards.  The  greatest 
difference  between  government  arid  commercial 
standards  on  any  given  program  is  that  government 
documentation  is  more  consistent  in  format,  and 
includes  additional  documentation  required  based 
upon  the  support,  concept  planned. 

Coiiniierc  i  a  1  Acqinsi  t  i  on 

When  a  coinaierciu  1  AID  is  procured,  there  are 
differences,  compared  to  Air  force  procurement s  , 
in  the  acquit i t ion  methods  used  as  well  as 
documentation  standards.  Where  the  Air  force 
requires  a  build-up  of  documentation  from 
contract  award  to  test,  commercially  only  the 
functional  design  definition  find  math  modeling 
details  are  documented  during  development. 
Informal  reviews  aie  held  throughout  the  develop¬ 
ment  process  with  design  discussions  recorded  in 
formal  minutes.  At  the  end  of  the  contract.  Cl’S 
documentation  that  is  delivered  with  the  AID  is 
very  similar  to  Air  force  development  documen¬ 
tation.  Confidence  of  acceptable  CPS  documen¬ 
tation  at  the  end  of  the  program  is  based  upon 


the  advantage  all  commercial  aircraft  companies 
have  over  the  government.  That  is,  if  the  simu¬ 
lator  contractor  does  not  deliver  what  is  required 
within  cost  and  schedule,  the  commercial  aircraft 
company  can  easily  take  their  business  elsewhere. 

Commercial  Maintenance 


Commercial  airline  companies,  i.e.,  those 
companier  that  train  commercial  aviation  aircrews, 
(American,  United,  etc.),  have  a  stable  group  of 
simulator  operational  support  engineers  and  tech¬ 
nicians  that  accomplish  all  software  maintenance 
and  modifications.  Ihis  group  has  years  of 
experience  and  is  reasonably  well  paid;  thei afore, 
t(>e  companies  do  not.  have  a  major  problem  with 
turnover.  Attrition  is  handled  through  employment 
(ft  other  experienced  simulator  engi neers/ tech¬ 
nicians,  often  from  milit.aiv  avenues  or  engineers 
from  simulator  manufacturers .  Any  shortages  in 
t tie  commercial  documentation  as  delivered  is 
usually  updated  and  expanded  as  necessary. 

COMPUTERIZATION  OF  DOCUMENTATION 
WhdtJsJ_t._? 

There  are  many  different  ways  documentation 
can  be  computerized ,  from  digitally  storing  text 
and  graphic  images  to  writing  programs  to  generate 
the  documentation.  All  of  these  concepts  have 
been  created  to  save  the  large  expense  that  soft¬ 
ware  documentation  adds  to  a  program.  Some 
computer  vendors  are  developing  a  lire  of  commer¬ 
cially  available  software  that  helps  the  user 
develop,  test,  document,  and  control  application 
programs.  Software  houses  and  simulator  companies 
have  developed  documenting  programs  or  concepts 
which  include  the  computerization  of  software 
document  generation.  Some  examples  of  types  of 
programs  include  utilizing  program  design  lan¬ 
guages  to  write  all  the  narratives  and  descrip¬ 
tions;  flow  generators  that  can  produce  a  diagram 
from  functional  block  level  down  to  a  detailed 
flowchart;  u.id,  with  a  small  modification,  the 
documentation  of  the  system  hardware.  Probably 
ninety  to  ninety-five  percent  of  all  software 
documentation  or  system  documentation  can  be 
computer i zed  to  some  degree.  The  remaining  five 
to  ten  percent  is  of  the  type  that  is  just  mere 
cost  effective  when  accompl i shed  manually. 

Contractor  Costs 

If  a  contractor  made  the  initial  etfort  ot 
developing  a  documentation  system  internally  for 
use  on  future  programs;  he  could  reduce  documen¬ 
tation  costs  approximately  fitty  percent.  An 
extensive  system  could  cut  documentation  costs  to 
less  than  a  dime  a  page,  just  implementing  con¬ 
cepts  t ha t  have  been  refined  in  the  last  few 
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years,  and  are  being  discussed  at  many  of  the 
national  computer  conferences .  The  contractor 
costs  would  consist  only  of  development  costs  of 
the  software  and  any  residual  tasks  for 
implementing  the  package  on  different,  machines, 
which  could  be  minimal  if  portability  were  a 
design  constraint. 

Government  Costs 


To  implement  some  type  of  computeri ra¬ 
tion  for  documentation  many  questions  need  to 
be  answered.  Would  it  be  more  cost  effective 
to  standardize  on  one  approach  and  require  usage 
of  that  standard  for  all  programs,  or  to  provide 
a  standard  program  and  then  require  usage  of 
that  program?  A  study  effort  to  define  what 
the  standard  should  be,  with  coordination  between 
AFSC,  AFLC,  Navy,  and  users,  would  probably  +ake 
one  year.  An  alternate  approach  to  reduce  support 
costs  would  be  to  purchase  a  "documentation 
system"  and  digitally  store  all  documentation 
tu  a  specified  format,  compatible  with  "he 
hardware  system.  I  ven  this  generates  questions 
as  to  how  many  hardware  systems  would  be  bought 
and  by  whom?  [ach  individual  program  could 
require  computerization  to  the  maximum  degree 
technically  feasible.  Costs  would  be  reduced 
by  paying  only  licensing  fees  for  previously 
developed  programs.  There  are  many  alternatives 
and  considerations,  too  many  to  name,  from 
which  the  government  can  choose. 

CONCLUSION.) 

Documentation  Standards 


One  of  the  largest  continuing  issues  in  the 
DOD  is  the  standardi zation  of  military  documen¬ 
tation  requirements  among  the  services.  Fach 
has  its  own  CPS  uocumen' a !  i ui»  standards,  with 
the  Army  using  th?  Navy  1644  standard  frequently. 
In  planning  a  reduction  of  documentation  costs 
high  priority  should  he  the  definition  of  a 
government  standard,  at  least  for  training 
levices.  Currently,  an  attempt  is  heinq  made 
to  define  this  standard.  With  this  t a s >■ 
accomo 1 i shed ,  a  comparison  against  commercial 
standards  should  be  performed  to  define  the 
documentation  gap  This  comparison  should 
identify  the  major  penalties  the  government  pays 
for  normal  maintenance  concepts  and  where  soft¬ 
ware  documentation  costs  can  be  cut.  Initial 
evaluation  shows  that  t here  is  no  major 
differences  between  CPS  documentation  procured 
for  an  ATD  with  plans  for  CIS  and  a  commercial 
ATD,  except  for  enforcing  contractor  produced 
standards . 

L o<j i  stj cs_  Support 

The  tie  between  the  skills  availabl  to  do 
software  maintenance  and  support  and  software 
documentation  is  far  too  direct  to  gloss  over. 

At  the  beginning  of  every  program  the  user's 
operational  and  maintenance  concept  and  stuffing 
for  the  new  ATD  must  be  the  primary  input  for 
documentation  planning.  If  a  CLS  concept  is 
being  planned,  then  mi  1  i  tary-sf  jrdarc  documentation 
is  unnecessary.  If  an  organic  concept  is  being 
planned,  commercial  standards  will  surely  fall 
short  of  what  is  needed  to  support  the  device. 

The  step  of  tying  user's  plans  with  documen¬ 


tation  requirements  is  one  of  the  most  important 
pre-Request  for  Proposal  activities. 

With  the  many  new  directions  being  received 
we  may  be  forced  into  a  (.IS  concept  for  all  future 
ATDs.  Although  there  is  user  resistance,  CLS 
should  force  a  reduction  in  government  CPS 
documentation  requirements,  as  well  as  its 
associated  costs.  Other  ways  of  reducing  govern¬ 
ment  CPS  documentation  requirements  are  tied  to 
the  limited  training  and  tour  length  of  ATD 
maintenance  technicians.  By  increasing  the 
quality  of  training  available.  Type  1  and  local 
university  courses,  extending  tour  lengths  and 
overlapping  transition  periods  when  duty  station 
changes  are  unavoidable  the  government  can  create 
a  maintenance  concept  close  to  that  of  the  major 
airlines.  However,  even  these  two  concepts 
would  not  resolve  the  largest  problem.  Software 
documentation,  even  by  commercial  standards, 
i*  complete,  is  too  costly. 

Documentation  Costs 


Fven  is  these  two  recommendations  are 
implemented  there  is  still  a  problem  of  high 
documentation  costs.  The  area  of  greatest  promise 
for  cost  savings  in  documentation  >s  in  the  area 
ot  computerization.  The  computer  p(oducts 
industry  and  computer  resources  institutes 
have  recognized  this  in  past  years.  Evidence 
of  this  can  be  seen  by  the  prominence  of  written 
and  presentation  material  at  many  computer 
conferences.  The  application  of  these  concepts 
should  directly  impact  the  acquisition  and  life 
cycle  support  costs  of  software  documentation 
for  all  government  training  devices.  If  applied 
carefully  many  existing  problems  should  be 
resolved.  The  actual  implementation  of  the 
techniques  should  be  accomplished  through 
validation  against,  a  minimum  government  documen¬ 
tation  standard  before  contract  award. 
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ABSTRACT 

“  Part  of  every  modern  cockpit  simulator  is  an  interface  between  the  analog 
world  of  the  cockpit  and  the  digital  world  of  the  central  simulation  computer(s). 
This  has  traditionally  been^a  complex  system  of  analog/digital  conversion 
circuitry  commonly  called  the  /^linkage'*’’. 

This  linkage  is  expensive  to  build  and  requires  modification  to  both 
the  hardware  and  central  computer  software  whenever  changes  are  made  to  the 
cockpit  instruments  and  controls.  It  also  imposes  a  significant  processing 
load  on  the  simulation  computer  in  that  much  polling  of  input  and  formatting 
of  data  are  required.  This  paper  describes  a  new  approach  that  solves  the 
problem  by  replicing  the  linkage  with  a  microcomputer  attached  to  each  major 
instrument /cont  rol  and  small  groups  of  minor  instruments/controls*  These 
instrument/control  computers  (ICC)  functionally  become  patt  of  the  instrument- 
/control  and  assume  all  simulation  of  that  device  that  is  independent  of 
other  activity.  The  ICCs  pass  data  to  the  central  computer  and  receive  data 
from  it  only  as  necessary. 


BACKGROUND 

The  control  of  an  aircraft,  and  for  that 
matter  almost  any  machine,  has  traditionally  been 
an  analog  function.  That  is,  the  pilot  moves 
the  main  controls  through  an  infinite  number  of 
positions  and  views  his  situation  via  indicators 
that  can  assume  an  infinite  number  of  positions 
on  the  face  of  a  dial.  True,  there  are  a  number 
of  important  devices,  such  as  switches  and  indicator 
lights,  that  are  discrete  in  nature.  And  in  modern 
cockpits  the  trend  is  to  more  and  more  discrete 
indicators.  such  aa  computer  driven  CRTs.  Despj.ce 
this  trend  the  key  controls  and  the  key  indicators 
are  likely  to  remain  analog  devices. 

In  the  early  days  of  aircraft  simulation 
the  heart  of  the  simulation  itself  was  an  analog 
device,  in  fact  it  was  an  analog  computer.  As 
the  technology  of  digital  compul  ,-rs  came  to  the 
fore  they  began  to  take  over  the  job  of  providing 
the  simulation  of  flight.  Although  they  were 
digital,  or  discrete,  in  nature  they  provided 
flexibility  and  cost  advantages  that  far  overshadowed 
their  shortcomings.  And  it  soon  became  apparent 
that  if  a  series  of  discrete  steps  were  small 
enough  and  quick  enough  they  would  appear  to  be 
continuous  or  analog  in  nature.  Sc  the  simulation 
problem  has  beer  taken  over  by  the  digits1  computer 
in  •  irtually  al.’  cockpit  simulators  in  recent 
years. 

Although  the  digital  computer  handled  the 
modeling  problem  quite  well,  the  controls  and 
instruments  had  to  remain  analog  if  they  were 
to  appear  at  all  realistic.  The  problem  of  the 
digita!  simulation  flight  model  on  one  hand  and 
the  analog  cockpit  on  the  other  was  handled  by 
constructing  an  elaborcte  conversion  mechanism. 
This  converts  analog  signals  flowing  from  the 
cockpit  to  digital  values  that  can  be  digested 
b>  the  computer,  and  translates  digital  values 
coming  from  the  computer  to  analog  voltage  levels 
or  synchro  signals  that  drive  h*  instruments. 
This  mechanism  is  commonly  known  and  is  hereafter 
referred  to  in  this  paper  as  the  "linkage"  (Fig.  1). 


Figure  1.  Convent ionn )  Simulator  A.vhlteet ure 


This  linkage,  aside  from  its  function,  has 
many  interesting  characteristics.  Most  of  them 
are  undesirable.  This  paper  outlines  a  new  approach 
to  the  problem  of  connunicat ing  between  the  analog 
cockpit  and  the  digital  simulation  computer  that 
does  awsy  with  the  linkage  and  its  associated 
problems . 
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THE  PROBLEM 

The  linkage  is  expensive  and  inflexible. 
This  inflexibility  . ukes  it  even  more  expensive 
when  modifications  to  the  prototype  aircraft  must 
be  reflected  in  the  simulator. 


Hardware 

The  linkage  as  found  in  most  cockpit  similators 
is  a  set  of  hardware  components,  mostly  A/D  (Analog 
to  Digital)  and  D/A  (Digital  to  Analog)  conversion 
circuitry.  The  linkage  is  built  differently  for 
each  different  aircraft  that  is  being  emulated. 
This  rrflects  the  fact  th3t  each  different  aircraft 
has  a  different  set  of  instruments  and  controls, 
although  many  of  the  instruments  aid  controls 
themselves  may  be  exactly  alike.  For  example, 
it  is  not  at  all  uncommon  for  different  aircraft 
to  have  the  same  make  and  model  of  altimeter. 
In  the  real  aircraft  this  commonality  has  all 
sorts  of  advantages,  such  as  reduced  parts  inventory, 
lower  manufacturing  costs,  etc.  But  in  the  building 
of  the  linkage  this  cannot  be  taken  advantage 
of  because  the  linkage  must  match  the  set  of  instru¬ 
ments  as  a  whole.  That  is,  when  instruments  and 
controls  are  added,  deleted,  or  modified  in  the 
prototype  aircraft,  the  linkage  must  be  modified 
to  reflect  this.  While  simulator  manufacturers 
have  designed  the  linkage  hardware  to  be  relatively 
easy  to  modify,  this  modification  still  entails 
considerable  expense. 

But  this  cockpit  half  of  the  linkage  is  but  one 
side  of  the  story.  Another  tale  lies  with  the 
computer  side  of  the  linkage,  vhere  the  problem 
is  even  more  severe. 

The  hardware  connection  is  a  custom  built  interface 
between  that  particular  linkage  and  that  particular 
make/model  of  computer.  If  the  computer  is  to 
be  changed,  for  whatever  reason,  this  interface 
also  must  be  rebuilt  or  at  least  modified.  Efforts 
have  been  made  by  the  simulator  manufacturers 
to  decign  flexibility  into  the  linkage  to  reduce 
the  impact  of  changes  in  the  computer,  but  again 
this  is  a  compromise  of  necessity,  not  a  solution. 


All  the  problems  that  have  been  mentioned  thus 
far,  however,  deal  with  hardware  changes  and  their 
impact.  But  an  even  more  expensive,  and  less 
tractable,  problem  lies  with  the  software  associated 
with  the  linkage. 

The  information  that  flows  between  the  computer 
and  the  linkage  is  not  foot  pounds  of  force  on 
the  yoke,  nor  is  it  feet  of  altiti  ,  nor  is  it 
any  of  the  units  with  which  the  simulation  computer 
deals  in  its  modeling  calculations.  The  values 
that  actually  flow  are  voltage  levels  or  synchro 
signals  that  indicate  either  the  angular  position 
of  a  control,  or  the  angular  position  that  an 
instrument  needle  is  to  assume.  This  implies 
that  a  lot  of  converting  between  formats  is  going 
on.  And  while  many  conversions  are  straight  forward, 
most  of  them  are  not.  Let  us  examine  the  typical 
case  of  a  TACAN. 


The  TACAN  is  a  relatively  simple  (in  concept) 
navigation  device  found  in  most  military 
aircraft  that  tells  the  person  in  the  cockpit 
his  bearing  and  distance  from  a  fixed  transmitter 
that  he  has  tuned  in.  To  accomplish  this 
portion  of  the  s imul a tion  in  most  syst ems , 
the  following  sequence  of  events  occur  at 
least  five  (typically  20)  times  a  second: 

The  simulation  computer  asks  the  linkage 
for  the  voltage  value  indicating  the 
position  of  the  TACAN  channel  selector 
switch . 

The  linkage  reads  the  voltage  of  the 
line  associated  with  that  switch,  converts 
it  to  a  binary  value,  and  passes  it 
to  the  computer. 

The  computer  converts  the  voltage  to 
an  angular  position  and  associates  that 
position  with  a  channel  number. 

Using  that  channel  number  as  a  reference, 
the  computer  looks  up  the  geographic 
position  of  the  transmitter  associated 
with  that  channel  in  a  table  prestored 
in  th»  program. 

The  computer  then  looks  at  the  present 
position  and  altitude  of  the  aircraft 
and  calculates  the  bearing  and  distance 
from  the  transmitter  to  the  aircraft 
using  simple  trigonometry. 

The  computer  now  converts  the  bearing 
value  to  a  voltage  that  will  cause  the 
bearing  instrument  indicator  to  assume 
the  angle  associated  with  that  bearing. 

The  computer  then  converts  the  distance 
into  four  separate  voltage  values  that 
represent  indicator  positions  for  tenths 
of  miles,  miles,  tens  of  miles,  and 
hundreds  of  miles. 

When  all  this  is  done  the  computer  passes 
the  voltage  values  to  the  linkage  which 
converts  the  binary  values  to  voltages 
and  applies  them  to  the  lines  attached 
to  the  TACAN  bearing  and  distance  indicators. 

The  person  in  the  cockpit  knows  where 
he  is. 

The  sequence  of  events  above  illustrates 
several  points. 

Processing  Load.  The  amount  of  processing 
done  by  the  computer  is  a  key  factor  in  the  design 
of  any  -simulator.  Usually  the  processing  load 
is  quite  heavy.  The  above  sequence  illustrates 
what  must  be  done  to  service  one  instrument  for 
one  cycle.  In  a  typical  simulator  there  are  hundreds 
of  instruments  and  controls,  each  must  be  serviced 
several  times  a  second- 

The  computers  selected  to  perform  these  functions 
are  generally  of  relatively  high  capacity  and 
are  commensurately  expensive.  In  many  simulators 
several  computers  are  linked  together  and  share 
the  load.  Another  effort  generally  made  to  meet 
the  processing  requirement  is  a  specially  designed 
software  operating  system.  The  operating  system 


364 


tv%  .TO.wpiiip  ipa  ^'l«  hi 


».1* 


I 


is  either  written  completely  by  the  simulator 
builder  or  is  an  extensive  modification  of  one 
provided  by  the  computer  manufacturer.  Either 
alternative  if.  an  expensive  one.  The  application 
routiner-  'ire  written  very  carefully  and  exhaustively 
tested  to  irsure  that  they  not  only  function  correctly 
but  that  they  do  not  exceed  their  allotted  memory 
space  or  execution  time.  Meeting  time  (or  memory) 
constraints  is  generally  considered  the  single 
most  expensive  factor  in  sof:ware  costs. 

In  the  example  above,  the  amount  of  calculation 
actually  concerned  with  the  simulation  itself 
was  quite  small.  Most  of  the  processing  had  to 
do  with  converting  values  and  controlling  the 
linkage.  This  is  one  example,  but  it  is  representative 
of  what  a  simulation  computer  really  does. 

Modification  Impact  on  Software.  Whenever 
a  modification  is  made  to  the  cockpit,  perhaps 
the  addition  or  upgrade  of  an  instrument,  not 
only  does  the  linkage  have  to  be  modified,  as 
mentioned  earlier,  but  the  simulation  program 
will  have  to  be  modified  to  handle  the  change. 
This  is  a  very  common  occurrence  and  great  pains 
are  taken  by  the  programmers  to  ensure  that  any 
new  processing  involved  does  not  exceed  any  critical 
memory  or  timing  constraints. 

'HIE  SOLUTION 

Suppose  now  we  could  eliminate  the  linkage 
as  described  herein.  Along  with  this  we  could 
eliminate  the  requirement  to  control  it  and  the 
requirement  to  constantly  reformat  the  data  flowing 
between  the  computer  and  tht  linkage.  We  could 
-hen  concentrate  solely  on  the  true  simulation 
functions.  This  would  reduce  the  total  processing 
load  by  several  factors  anc*  with  this  reduction 
one  can  suddenly: 

Reduce  the  number  of  central  computers  in 

the  simulator. 

Utilize  less  expensive  computers. 

Use  standard  operating  systems. 

Simplify  development  of  application  software. 

Reduce  the  cost  of  building  and  maintaining 

a  simulator. 

The  rest  of  this  paper  explo  es  a  method 
of  doing  just  that . 

The  Concent 

Put  simply,  it  is  to  attach  and  devote  an 
entire  computer  to  each  major  instrument,  each 
major  control,  and  each  group  of  minor  instruments 
and  controls  in  the  cockpit.  Such  a  computer 
would  directly  move  the  indicators  on  an  analog 
instrument  based  on  digital  data  passed  to  it 
from  a  central  simulation  computer.  Anothei  would 
constantly  observe  the  position  of  an  analog  control 
and  periodically  send  information  concerning  it 
to  the  central  computer. 

Each  of  these  computers  would  be  complete 
in  itself,  independent  of  the  otters,  and  dependent 
on  the  central  computer  cnly  for  data.  Data  passed 
between  the  computers  would  be,  of  course,  in 
digital  format  and  expressed  in  units  that  the 
simulation  pro'ram  naturally  deals  with. 


The  computers  coupled  to  the  ins  ruments 
and  controls,  henceforth  referred  to  as  ICCs  (Instru¬ 
ment/Control  Computers)  must  be  complete  computers, 
with  a  central  processing  unit  (CPU),  memory  and 
I/O  ( input /output )  capabilities.  But  they  will 
not  be  the  large,  cabinet  mounted  devices,  that 
one  usually  associates  with  computers.  They  will, 
with  few  exceptions,  be  contained  on  a  single 
chip  and  they  will  be  identical.  The  difference 
will  be  the  program  within  them.  A  subsequent 
section  provides  more  detail  on  the  characteristics 
of  these  computers  and  the  method  used  to  communicate 
between  them. 


ICCs  driving  instruments  will  primarily  format 
and  convert  the  data  as  received  from  the  central 
computer  and  directs  its  output  to  the  instrument. 
They  may  also  provide  some  of  the  more  rudimentary 
simulation.  For  example,  the  ICC  driving  a  heading 
indicator  may  provide  a  dampening  or  coasting 
effect.  This  would  lower  the  num!  er  of  updates 
required  by  the  central  computer  but  would  maintain 
smoothness  and  accuracy.  Another  such  situation 
would  be  the  ICC  monitoring  the  control  yoke/stick. 
In  present  simulator  design  the  central  computer 
periodically  polls  the  control  and  receives  back 
a  set  of  values  indicating  its  present  position. 
An  ICC  would  monitor  the  position  constantly  and 
periodically  report  back  rot  only  its  position, 
but  could  easily  determine  and  report  the  amount 
of  change,  direction,  and  velocity.  These  values 
are  now  computed  by  the  central  computer. 


An  even  higher  level  of  simulation  could 
be  handled  by  the  ICC  provided  t.iat  the  function 
to  be  simulated  is  relatively  isolated.  Take 
again  the  example  of  the  TACAN  mentioned  earlier. 
Almost  the  entire  simulation  problem  for  that 
device  could  be  handled  by  the  ICC.  That  ICC 
coulo  monitor  the  channel  selector  input,  determine 
the  position  and  ch  racteristics  of  the  transmitter 
from  a  table  stored  in  its  own  memory,  compute 
bearing  and  range,  and  drive  the  indicators. 
The  only  requirement  of  the  central  computer  in 
this  case  would  be  to  periodically  transmit  the 
aircraft's  position  and  altitude  to  the  ICC. 


Although  this  paper  is  primarily  concerned 
with  analog  functions,  it  should  be  mentioned 
that  this  concept  is  just  as  applicable  to  discrete 
functions.  That  is,  an  ICC  can  handle  switch 
inputs  and  discrete  outputs  as  well  it  can  analog 
values.  The  major  benefit  in  this  regard  is  that 
the  ICC  can  poll  the  discrete  devices  for  input 
and  only  report  changes  to  the  central  computer, 
thereby  relieving  the  central  computer  of  a  time 
consuming  task. 


MvanXaXJJ 
There  are  many. 

lower  Pppc easing  Load.  The  main  one,  already 
treated,  is  to  simplify  and  reduce  the  processing 
required  of  the  main  computer.  This  will  not 
be  dwelt  upon  further  except  to  say  that  the  processing 
load  will  be  a  small  fraction  o,  what  it  is  now. 
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Modularity.  Another  important  benefit  revolves 
around  the  concept  of  modularity.  For  purposes 
of  this  discussion  let  us  define  modularity  as 
that  quality  that  permits  us  to  add,  delete,  or 
modify  pieces  of  a  system  without  effecting  other 
pieces,  or  the  system  as  a  whole.  Perfect  modularity 
is  never  achieved  in  any  system  that  has  interrelated 
components,  but  modularity  is  generally  achieved 
to  some  degree. 


The  benefits  of  modularity  for  a  system  such 
as  a  simulator  reside  on  two  areas.  The  first 
is  cost  of  modification.  If  one  can  add,  change, 
or  delete  a  component  without  worrying  about  what 
effect  such  a  change  will  have  on  other  parts, 
the  cost  will  be  much  less  than  it  might  otherwise 
be.  This  is  generally  accepted  and  will  not  be 
elaborated. 


The  second  major  benefit  of  strong  modularity 
is  simpler  system  integration.  The  more  modularity 
that  the  designer  can  achieve  the  less  concern 
he  will  have  with  interference,  timing  constraints, 
resource  conflicts,  and  the  like.  Also  the  system 
implementor  will  have  to  spend  less  time  testing 
the  system  to  verify  with  confidence  that  it  performs 
as  intended.  All  of  this  translates  directly 
into  lower  total  costs.  This  author's  primary 
professional  activity  for  the  last  ten  years  has 
been  in  the  design  and  implementation  of  various 
large,  real-time,  process  control  systems.  It 
has  been  his  experience  that  approximately  one 
third  to  one  half  of  the  total  system  development 
effort  has  been  in  the  integration  and  testing 
phase . 


If  one  can  d-vote  &:i  entire  computer  to  one 
instrument  or  control,  many  integration  problems 
are  not  just  solved,  they  simply  do  not  exist. 
For  example,  if  one  can  program  one  of  these  computer, 
to  drive  the  TACAN  display,  one  can  test  the  function 
by  sending  the  present  aircraft  position  and  altitude 
to  it  and  observe  the  display.  It  makes  no  difference 
whether  the  program  was  written  by  a  novice  or 
an  experienced  programmer.  It  makes  no  difference 
what  the  make  or  model  of  the  central  computer 
is.  The  system  designer  does  not  have  to  be  concerned 
about  how  much  of  the  total  processing  time  or 
the  total  memory  that  the  TACAE  simulation  routines 
will  consume.  In  fact,  it  makes  no  difference 
what  aircraft  is  being  simulated.  It  will  function 
the  same  in  any  cockpit  simulator  that  has  that 
particular  TACAN. 


Also,  if  the  TACAN  display  in  the  simulator 
is  changed  for  any  reason,  only  the  program  in 
the  ICC  driving  it  need  be  replaced.  Such  changing 
will  have  absolutely  no  effect  on  the  central 
computer  or  its  program.  In  a  later  section  it 
will  be  shown  that  in  some  cases  it  will  also 
be  possible  to  add  instruments  to  the  cockpit 
without  modifying  the  central  computer  or  its 
program. 


A  few  words  should  be  said  about  the  capacity 
and  power  of  these  computers.  The  ICCs  as  envisioned 
here  have  me  lest  capabilities  compared  to  those 
driving  most  simulators  today.  However,  they 
are  quite  comparable  in  power  to  thv.se  available 
iust  a  few  years  ago.  And  when  one  considers 
that  ..hey  will  be  required  to  handle  just  one 
or  a  few  functions,  their  speed  and  memory  capacities 
will  be  more  than  adequate.  Ard  although  it  is 
envisioned  that  all  ICCs  will  be  alike,  they  need 
not  be.  A  more  powerful  version  could  be  inserted 
to  handle  a  particular  cockpit  device.  Its  only 
requirement  is  that  it  observe  the  same  conmunications 
protocol . 

Communicat ion . 

Several  references  have  been  made  to  passing 
dat3  between  the  ICCs  and  the  central  computer. 
Without  a  consistent  and  well  thought  ojt  convention 
in  this  area  all  the  benefits  mentioned  above 
will  not  accrue. 

There  are  several  ways  that  the  ICCs  and 
the  central  computer  can  be  linked.  The  one  mentioned 
here  appeals  the  least  costly  and  easiest  to  imple¬ 
ment  . 

The  method  presented  here  is  simply  the  serial 
linking  (daisy  chair.)  of  all  the  ICCs  on  a  single 
line  from  the  central  computer  (figure  2).  The 
electrical  standard  is  a  subset  of  the  common 
commercial  RS-232.  The  ICCs  are  arranged  in  a 
loop  network,  where  the  input  line  of  an  ICC  is 
connected  to  the  output  line  of  the  upstream  ICC. 
The  input  line  of  the  first  ICC  is  connected  to 
the  transmit  line  of  the  central  computer's  serial 
port  and  the  output  line  of  the  last  ICC  is  connected 
to  the  receive  line  of  the  port.  Salient  features 
of  this  scheme  are: 

Data  would  remain  in  binary  format,  preferably 
in  the  same  notation  used  by  the  central 
computer's  main  simulation  program. 

Dat  would  be  packaged  in  messages.  Each 
message  would  have  ■  header  containing  an 
address,  message  identifier,  length  indicator; 
the  data  of  interest;  and  a  trailer  containing 
a  sirpie  checksum  or  other  error  checking 
mechanism. 

Messages  from  the  central  computer  to  the 
ICCs  would  be  either  of  a  broadcast  nature 
(with  data  of  interest  to  more  than  one  ICC) 
or  would  be  addressed  to  a  particular  ICC. 
The  address  of  the  ICC  would  be  its  position 
on  the  chain.  If  an  instrument  and  its  associated 
ICC  require  only  broadcast  data,  it  could 
be  added  to  the  cockpit  without  modification 
of  the  central  computer. 

In  this  scheme  the  central  computer  would 
initially  transmit  a  message  to  the  first  ICC 
cn  the  chain.  The  ICC  would  immediately  retransmit 
it  to  the  next  station  downstream,  no  matter  what 
message  it  is  or  for  whom  it  is  intended.  Each 
of  the  ICCs  would  do  this  with  every  message  no 
matter  what  position  it  held  in  the  chain.  As 
part  of  the  retransmission  an  ICC  would  decrement 
the  address.  After  the  retransmission  wns  complete, 
the  ICC  would  look  at  the  address.  If  the  address 
value  is  zero,  the  ICC  would  know  ’ha:  :he  message 
was  intended  for  it  and  would  act  upon  it. 
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Figure  2.  Multi-Microcomputer  Simulator 
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The  retransmitted  messages  would  eventually 
get  back  to  the  central  computer  which  could  be 
compare  them  with  what  had  beea  sent.  This  is 
tb  most  foolproof  of  error  checking  schemes. 
This  might  be  useful  in  some  very  critical  applica¬ 
tions,  but  the  communication  environment  inside 
a  simulator  is  quite  benign  and  communication 
errors  are  not  likely  to  be  a  problem. 

ICCs  wishing  to  originate  messages  on  this 
net  would  simply  transmit  them  whenever  they  are 
not  retransmitting  other  messages.  The  initial 
address  word  in  this  case  would  be  zero.  By  the 
time  the  message  reached  the  central  computer 
the  address  slot  would  have  a  negative  vrlue  corre¬ 
sponding  to  the  ICC's  position  on  the  net.  This 
technique  eliminates  the  complication  of  a  master 
ICC  on  the  net  tc  arbitrate  requests. 

Part  of  the  communications  software  package 
in  each  ICC  would  be  a  watch  dog  timer  that  would 
originate  a  trouble  message  if  it  had  not  received 
A  message  within  a  specified  period  of  time. 
Whenever  such  a  message  were  received  by  the  central 
computer,  it  would  know  and  could  inform  an  operator 
that  an  ICC  had  failed,  and  from  the  address  value 
of  the  message  could  determine  which  ICC  had  failed 
(it  is  the  one  upstream  from  the  ICC  thu  reported 
it). 

This  communications  scheme  has  several  inter¬ 
esting  characteristics: 

Simplicity.  It  uses  the  most  common  and 
universal  electrical  protocol.  It  is  familiar 
to  most  programmers  and  engineers  and  is  supported 
by  virtually  all  existing  operating  systems. 


Speed /Capacity .  As  networks  generally  go, 
this  nmthod  is  not  conducive  to  moving  great  amounts 
of  data  quickly.  But  it  should  be  adeqiMte  for 
this  application.  To  demonstrate  this  let  us 
look  at  what  a  typical  communications  load  might 
be.  Let  us  assume  that: 

The  transmission  rate  is  188  kilobaud.  The 
ICCs  as  described  later  can  obtain  this  rate. 
With  10  bits  per  character  we  have  a  character 
rate  of  18,800  characters/second. 

The  central  computer,  running  the  main  flight 
dynamics  package,  broadcasts  a  message  containing 
key  data  such  as  current  altitude,  speed, 
heading,  pitch  angle,  roll  angle,  vertical 
speed,  angle  of  attack,  and  X-Y  position 
five  times  a  second.  Each  of  these  values 
can  be  held  within  32  bits.  The  entire  data 
portion  of  the  message  would  then  be  288 
characters.  With  header  and  trailer  characters 
let's  assume  this  message  to  be  300  characters 
long . 

The  ICC  monitoring  the  yoke/stick  and  rudder 
pedals  reports  position,  delta  position, 
and  velocity  of  each  of  these  controls  20 
times  a  second.  Each  of  these  values  can 
be  held  in  32  bits.  The  data  portion  of 
this  message  would  then  be  192  characters 
long.  With  the  tamo  header  and  trailer  the 
entire  message  would  use  204  characters. 

The  remaining  traffic  would  reflect  asynchronous 
events  such  as  the  pilot  flipping  switches 
or  the  central  computer  lighting  various 
indicators.  Let's  assume  that  this  information 
requires  100  characters/second. 

The  total  communications  load  would  then 
be : 

( 300*5)  ♦  (204*20)  ♦  100  ■*  5680  chars/second. 

At  first  glance  it  may  appear  that  the  time 
required  to  retransmit  a  message  will  cause  significant 
delays.  If  it  takes  100  taicrc seconds  to  relay 
a  word  in  each  ICC,  the  first  word  of  a  message 
will  reach  the  10th  ICC  in  cne  ®f  1  lisecond .  This 
is  not  significant  in  light  of  the  bisic  transmission 
rate . 

A  multi-drop  network  based  on  a  high  speed 
bus,  such  as  Xerox's  Ethernet,  could  also  be  used. 
There  would  be  no  ret rsnseii ss  ion  delay  and  the 
capacity  would  be  higher  i  would  the  cost  of 
hardware  and  software  needed  to  support  it.  From 
the  viewpoint  of  required  capabilities,  such  an 
effort  does  not  appear  to  be  justified. 


Kel  isbj) ipy.  Stringing  all  the  ICCs  together 
in  serial  fashion  has  the  inherent  drawback  that 
the  failure  of  one  of  the*  will  effect  the  whole 
system.  Ceneraliy  system  designers  avoid  such 
situations.  But  the  simplicity  of  the  ICCs  will 
provide  high  inherent  reliability  in  each  of  them, 
the  high  modularity  of  the  system  will  make  them 
easy  to  replace,  ard  the  failure  reporting  mechanism 
mentioned  above  will  pinpoint  the  failed  unit. 
Based  on  this  the  complication  and  expense  of 
a  less  vulnerable  scheme  does  not  appear  to  be 
warranted . 
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The  Computer . 


In  order  to  make  this  whole  concept  viable 
the  computer  on  which  the  ICC  is  based  must  do 
very  small,  very  reliable,  and  very  cheap.  The 
historic  trend  in  computer  development  has  be<’n 
in  this  direction.  But  only  recently  has  development 
come  to  the  point  where  it  is  viable  to  devote 
an  entire  computer  to  each  instrument. 

The  forerunner  to  the  computer  to  be  used 
here  is  the  microprocessor.  This  had  a  very  signi¬ 
ficant  impact  on  the  industry  and  in  the  world 
in  general.  The  most  notable  result  is  the  home/hobby 
computer.  Capable  as  they  were,  microprocessors 
usually  consisted  of  a  CPU  only  and  required  signi¬ 
ficant  support  circuitry  in  the  form  of  clock 
chips,  I/O  chips,  interrupt  chips,  memory  and 
the  like.  Now  development  has  reached  the  point 
where  all  of  these  functions  are  combined  or  a 
single  low-cost  chip. 

These  chips  are  just  now  hitting  the  market. 
Zilog,  Inc.  offers  the  Z8,  Intel  has  the  MCS-51 
series  of  chips,  and  Texas  Instruments  will  be 
offering  such  a  chip  sometime  in  1982.  Of  the 
two  currently  available,  the  Intel  offering  is 
newer  and  has  more  of  the  capabilities  required 
for  this  application.  Some  of  its  important  character¬ 
istics  are  indicated  below: 

Four  KB( k iloby t es  )  of  on-chip  memory.  128KB 
addressable  memory  using  auxiliary  chips. 

Single  5-volt  power  source  requited. 

Five  interrupt  sources  available  with  automatic 
interrupt  vectoring.  Two  lr>vel  int^Tiipt 
priority. 

Two  interval  counter/tiraers . 

All  timer,  interrupt  support  functions,  etc  on 
board  the  chip. 

Built-in  serial  input/output  line  30  discrete 
input/output  lines. 

Very  powerful  processor  that  includes: 

Hardware  multiply/divide  (four  micro¬ 
seconds)  . 

Four  sets  of  eight  general  purpose  registers. 

Stack  instructions. 

Multiple  addressing  modes. 

Boolean  subprocessor  for  bit  manipulation. 
Less  than  $25  in  single  quantities. 

FUTURE  EXPANSION 

This  concept  can  be  quite  easily  expanded 
to  include  more  thrn  just  the  devices  in  the  cockpit. 
Visual  systems,  motio  platforms,  and  instructor /oper¬ 
ator  stations  (I0S)  could  be  included  on  the  same 
loop  (fig.  3)  or  a  separate  loop.  This  would 
establish  a  de  facto  standard  interface  for  these 
components  to  whicn  manufacturers  of  these  major 
subsystems  could  design. 


CENTRAL 

COMPUTER 


F f gure  3.  Modular  Cockpit  Simulator 
Architecture 

There  is  a  movement  in  the  simulator  world 
to  expand  the  instructional  features  of  simulators 
to  include  scenario  generation,  performance  evaluation, 
graphic  displays,  br ie f ing/debr ie t ing ,  replay, 
record  keepirg,  and  the  like.  Such  facilities 
usually  require  separate  computer  systems.  An 
ir.terface-  mechanism  based  on  the  concept  outlined 
in  tnis  paper  would  permit  the  addition  of  these 
facilities  without  serious  impact  on  the  rest 
of  the  system.  That  is,  it  would  be  possible 
to  exchange  simple  IOSs  with  sophisticated  ones 
or  vice  versa,  depending  on  the  role  of  the  simulator, 
features  desired,  availability  of  funds,  etc. 

SUMMARY 

Once  an  ICC  is  built,  programmed  and  attached 
to  an  instrument,  it  is  a  part  of  that  instrument 
forever.  It  can  be  used  without  modification 
in  any  simulator.  On  the  other  hand  a  modified 
ins  rument/ICC  can  replace  an  existing  one  without 
effecting  the  central  computer  and  its  software. 

The  strong  modularity  inherent  in  this  approach 
simplifies  system  design,  eases  system  integration, 
and  reduces  required  testing.  The  simulation 
and  other  processing  performed  by  the  ICCs  will 
greatly  reduce  the  processing  load  on  th-.*  central 
simulation  compute! . 

The  ability  to  interface  dissimilar  subsystems 
will  permit  the  system  designer  to  pick  and  choose 
major  components  from  different  manufacturers 
to  make  up  a  training  system  tailored  to  its  functional 
role. 
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ABSTRACT 

‘This  paper  provides  an  evaluation  of  computer  configurations  that  might  be  applied  to  a 
single  large  simulator  or  a  complex  of  simulators.  The  analysis  indicates  that  minicomputers 
or  the  recently  introduced  super  minicomputers  offer  the  best  approach  to  implementation  of 
most  simulators.  The  very  large  computer  designed  for  high  throughput  in  a  multiprogramming 
environment  is  not  cost  effective  for  the  simulator  application,  and  the  yet  unsolved  problems 
in  partitioning  the  simulator  computations  for  use  in  a  distributed  microprocessor  network 
outweigh  the  saving  in  hardware  cost.  More  important  than  the  results  of  th’s  study  is  the 


method  o'1  analysis  developed  for  evaluation  ■ 
The  procedure  presented  can  be  used  to  address 
such  variables  as  reliability,  mean  time  to 
use  in  evaluating  computer  implementation  for 

INTRODUCTION 

The  need  for  a  broad  analysis  of  computer 
architecture  is  the  result  of  procurement  under 
Office  of  Management  and  Budget  Circular  A-109. 
This  procurement  procedure  gives  vendors  a  w.der 
latitude  in  proposing  innovative  systems.  At  the 
same  time,  it  places  upon  the  government  a 
requirement  to  evaluate  these  proposals  to 
determine  whi^h  vendor's  approach  is  best.  The 
purpose  of  this  study  is  to  provide  reference 
material  and  guidelines  in  one  of  the  critical 
areas  (computer  r-ystems)  to  be  evaluated.  Although 
the  specifics  used  in  the  example  are  taken  from  a 
specific  trainer,  the  approach  is  applicable  to 
flight  trainers  in  general  and  the  ideas  are 
related  to  ar.y  kind  of  trainer. 

This  study  addresses  the  problem  of  what 
computer  configuration  should  be  used  in  the 
computer  complex  for  a  multiple-cockpit  flight 
trainer.  It  considers  the  characteristics  or  each 
generic  class  of  computers,  with  no  attempt  made  to 
assess  the  advantages  and  disadvantages  of  the 
manufacturers ’  products  within  the  class.  Computer 
architectures  for  performing  the  computation  and 
control  functions  required  are  evaluated  to 
determine  the  advantages  and  disadvantages  of  each. 

Cockpit  Conf igu r a t i o n 

The  baseline  cockpit  configuration  selected 
r  the  analysis  is  the  representative  simulator 
suite  described  by  the  VTXTS  technical  team  (1j. 
This  training  suite  consists  of  six  Cockpit 
Procedures  Trainers  (CRT 'SI,  fifteen  Operational 
Flight  Trainers  (OFT'S)  without  visual  systems, 
nine  OFT'S  with  visual  systems,  and  six  Air  Combat 
Maneuvering  Trainers  (ACNT'Sl.  The  total  training 
suite  I.-  divided  into  three  identical  cockpit 
arrangements .  Figure  1  shows  one  of  these 
divisions  with  its  computer  complex. 

COMPUTER  PERFORMANCE 

Assess',- a  computer  system’s  ability  to  solve 
a  problem  s  a  critical  part  of  evaluating  various 
computer  areni tectures .  The  method  selected  is  to 
specify  the  problem  to  be  solved  by  the  computer 
complex  in  terms  of  computer  performanc e ,  number  of 


if  computer  configurations  in  simulator  systems, 
other  configurations  and  different  values  of 
repair  and  cost.  The  method  is  recommended  for 
specific  simulator  procurements.. 

Jk 

■-  P  T  j  ‘  rpr 


M  -  '  F  X 


Figure  1.  VTXTS  Cockpit  Arrangement 

instructions  to  be  executed  during  a  given  time, 
and  storage  requi  rerrents  . 

The  capability  of  a  given  computer  to  solve 
the  problem  is  based  upon  an  A.erage  Instruction 
Execution  Time  (AIET\  a  figure  of  merit  determined 
by  the  hardware  instruction  execution  times  of  the 
instructions  w°ighted  by  a  per  cent  .se  factor. 
Two  sets  of  weighting  factors  were  used,  one  for 
Operational  Flight  Trainers  (OFT’sl  and  one  for 
Weapon  System  Trainers  (WST’s'1,  based  upon  a 
composite  of  actual  instruction  counts  from  several 
existing  >°FT'S  and  WST's. 

Table  1  shows  AIET's  for  various  computer 
systems  for  the  two  instruction  mixes.  Results  for 
the  minicomputers  (Digital  PDP  11 '70,  Systems  SEL 
??/77,  and  Perki n-Elrr.er  PE  3<?^0)  are  taken  from  a 
report  [2],  The  performance  of  the  other  compute'-:? 
is  estimated  by  comparing  their  speed  to  the  sp^ed 


369 


rA‘<:>  A  Vr  cA  INrV'f ’■ ' TI  N  fe*fe* 't’/T  N 


AVfsA’.f  r:  •.  n  t:**f 


; :  'A  :so  ■■ 


of  the  minicomputers  for  a  subset  of  the  total 
instruction  repertoire.  The  Amdahl  computer 
performance  is  estimated  from  the  ratio  of  ADD  and 
MULTIPLY  instruction  execution  times  on  the  Amdahl 
UlOV/l  [31  compared  to  those  of  the  minicomputers. 
The  performance  of  the  microcomputers  (Motorola 
MC68000,  Zilog  Z8000,  and  Intel  8086)  is  estimated 
by  comparing  the  instruction  execution  speeds  given 
in  a  comparison  of  the  microprocessors  [4]  to  the 
minicomputer  instruction  execution  speeds. 
Capability  of  the  newest  entry  (System  32/87)  is 
estimated  from  the  ratio  of  performance  on  the 
British  Whetstone  benchmark  program  for  the  Model 
32/87  and  the  Model  12/11  given  in  the 
manufacturer's  data  sheet. 

Computer  Classes 

The  computers  considered  have  been  div*  ' 
into  four  classes  (Large  Computer, 
Minicomputer,  Minicomputer,  and  Microcomputer) 
un  appropriate  execution  speed  has  been  assigns 
each  class.  The  computers  are  divided  into 
classes,  because  the  purpose  of  this  study  is  to 
determine  computer  configurations  in  terms  of  these 
classes  and  not  to  distinguish  between  the  products 
of  various  manufacturers  within  a  class.  The 
AIET's  shown  in  Table  1  were  converted  into  the 
nominal  instruction  execution  speeds  for  each  class 
shown  in  Table  2. 

The  distinction  between  the  Large  Computer  and 
the  Super  Minicomputer  requires  some  explanation. 
Although  tne  examples  presented  differ  in 
computation  speed,  this  is  not  the  criterion  that 
distinguishes  the  two.  The  sup^r  minicomputer  is 
an  improved  32-bit  minicomputer  with  an 
architecture  which  results  in  3  MIPS  or  greater 
speed  in  executing  instructions.  The  large 
computer  is  one  designed  for  use  in  multi-us**r 
environments,  both  data  processing  and  scientific 
computation.  It  has  a  high-speed  arithmetic 
capability,  but  it  also  has  features  such  as 
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ClWFMTER  TYPE 

’APARILITY 

■’"'V'  Instructions  Seoon.1' 

ca-ge  Computer 

A 6, 000 

Minicomputer 

900 

Super  minicomputer 

r,  t  000 

Microcomputer 

000 

multiple  I/O  channels  and  sophisticated  operating 
systems  designed  for  the  multiprogramming 
environment.  Normally,  the  operating  system  for 
the  large  computer  is  not  designed  "or  time 
critical  tasks  in  the  frame  considered  in  real-time 
flight  simulators. 

STORAGE  AND  PROCESSING  REQUIREMENTS 

The  storage  and  processing  requirements  shown 
in  Table  3  were  derived  from  available  data  on 
trainers  in  the  Navy  inventory.  This  table 
provides  a  summary  of  the  minimum  and  maximum 
storage  and  processing  requirements  for  each  type 
of  simulator  and  shows  the  processing  and  storage 
requirements  assumed  for  this  analysis.  The 
complete  data  from  which  this  summary  is  made  are 
available  in  a  report  [5]. 
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COMPUTER  CONFIGURATIONS 


Tne  computer  system  for  a  CPT  is  usually  a 
single  computer/processor  configuration  with 
private  memory,  peripheral  equipment,  mass  storage 
memory  and  cockpit  input- output  conversion 
equipment  (linkage).  The  computer  sytem  of  a 
modern  OFT  or  ACMT  is  usually  a  multiprocessor 
configuration  with  private  memory,  shared  or  common 
memory,  peripheral  equipment,  mass  storage  memory, 
and  cockpit  input/output  data  conversion  equipment. 
This  system  is  typical  of  the  conventional 
master-slave  computer  system  configuration  [6,73. 

For  a  complex  of  trainers,  the  designer  has  a 
much  wider  choice  of  computer  arrangements.  The 
configurations  considered  range  from  a  single  large 
computer  to  a  distributed  system  of  38 
microcomputers.  In  a  multiprocessor  configuration, 
one  processor  is  designed  as  the  master  unit  which 
controls  the  training  system.  The  master  processor 
usually  contains  the  software  for  instrjctor 
station  functions,  data  recording/playback, 
simulator  modes  of  operation.  The  slave  processors 
contain  the  software  for  flight,  engines, 
accessories,  communications  -'navigation ,  weapons  and 
all  the  ether  simulation  functions. 

Minicomputers 

Two  different  minicomputer  configurations  are 
considered  in  this  analysis.  The  simpler  in  terms 
of  implementation  is  shown  in  Figure  2.  This  is  an 
attempt  to  use  a  one-on-one  match  of  computer  to 
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Figure  2.  Dedicated  Minicomputer  Systems 


cockpit,  modified  to  take  care  of  gross  mismatches 
in  the  simulator  requirements  and  the  computer 
capabilities.  For  the  OFT's  the  match  in 
capabilities  and  requirements  allows  the  one-on-one 
arrangement.  Since  the  CPT's  use  less  than  half  of 
the  capability  of  a  single  computer,  one  computer 
is  used  to  drive  two  CPT's.  The  ACMT 
implementation  introduces  the  opposite  problem. 
Since  a  single  computer  does  not  have  enough 
capability  to  perform  tp  computations  required, 
two  computers  must  be  user  t^r  each  ACMT. 


The  alternative  to  a  one-on-one  arrangement  is 
to  use  multiple  computers  to  drive  multiple 
cockpits.  Figure  3  shows  this  concept  applied  to 
the  entire  complex  of  cockpits.  TMs  results  in 
considerable  saving  in  the  number  of  computers 
required,  reducing  the  13-computer  configuration 
shown  in  Figure  2  to  only  8  computers.  The 
disadvantage  of  this  approach  is  an  increase  in 
software  cost  and  complexity. 


K‘.  N 


Figure  4  shows  a  Super-Minicomputer 
configuration  for  the  cockpit  complex.  Use  of 
Super  Minicomputers  is  attractive  because,  unlike 
the  large  computers  described  next,  the  Super 
Minicomputer  provides  a  greater  increase  in 
processing  capability  than  the  increase  in  cost. 
The  reason  for  this  iJ  that  the  major  additional 
cost  is  in  the  trgh-speed  arithmetic  element  rather 
than  I/O  devices  or  sophisticated  hardware  for 
improving  the  computer's  performance  in  a  multiuser 
environment . 


Figure  U,  Super  Minicomputer  Configuration 


Large 


Computers 


A  single  large  computer  can  be  use>i  to  perform 
the  requir'd  computations  for  the  whole  cockpit 
complex.  The  major  effect  of  using  a  single  large 
computer  is  a  decrease  in  total  availability  of  the 
system. 


A  major  drawback  in  using  a  large  computer  for 
the  simulator  complex  is  its  cost.  In  general,  the 
cost  of  a  larger,  faster  computer  trows  more  than 
linearly  (i.e.,  a  computer  ten  times  as  fast 
usually  costs  more  than  ten  times  as  muchl .  For 
example,  the  Amdahl  H70V/,?  costs  about  25  times  the 
price  of  a  Perkin-Elmer  32^0  [3]  but  has  on]''  a  20 
to  1  advantage  in  computation  speed.  Use  of  a 
large  computer  cannot  be  discarded  in  all  cases, 
but  this  type  of  architecture  does  not  seem  to 
provide  any  advantage  in  the  the  present 
appl ication. 


Microcomputers 
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Figure  3.  Shared  Minicomputers 


A  microcomputer  configuration  for  the  cockpit 
complex  is  shown  in  Figure  5.  Use  of  multiple 
microprocessors  offers  saving  in  hardware  cost  over 
the  use  of  larger  computers,  0  mmer  and  Wyndle  [81 
estimate  the  cost  of  a  microprocessor  OFT  to  be 
little  more  than  one-half  the  cost  of  an  OFT 
implemented  with  minicomputers.  Their  analysis  of 
life  cycle  costs  shows  a  similar  saving  in  software 
and  maintenance  costs.  Tndeed,  the  multiple 
microcomputer  network  provides  a  number  of 
advantages  over  other  approaches.  Uowever,  these 
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Figure  5.  Dedicated  Microcomputer  Systems 


advantages  are  outweighed  by  its  cne 
disadvantage — such  a  system  has  not  yet  been 
developed.  Use  of  multiple  microcomputers  is 
likely  to  result  in  high  development  costs  and  an 
associated  high  risk  in  added  cost  and  in  schedule 
slippage. 

The  developmen*  cost  and  high  risk  o^  a 
microcomputer  system  result  from  problems  that  must 
be  solved  in  order  to  use  multiple  microcomputers 
effectively.  Three  major  steps  must  be  taken  in 
order  to  take  advantage  of  the  inherent  parallelism 
that  exists  in  many  applications. 

a.  Partitioning  of  the  problem  into  disjoint 
tasks . 

b.  Provision  for  centralized  control. 

c.  Development  of  a  run-time  structure  which 
provides  for  the  passing  of  system 
parameters  between  tasks  and/or  processors 
while  preserving  precedence. 


NAVTRAEQUIPCEN  has 

conducted 

an 

active 

research  program  in  this  area  since 

197*1 

[9, 10]. 

Implementation  problems 
incl ude : 

which  have 

been 

solved 

a.  The  development 

and  d  monstration 

of  a 

control  algorithm  for  WNW  microcomputers 
embodied  in  hardware,  firmware  and 
software.  This  control  algorithm  is 
identical  in  each  microcomputer  and 
functions  as  an  applications  task  manager 
(ATM)  for  distributed  control. 

b.  The  development  and  demonstration  of  ? 
distributed  cache  concept  whereby  the 
address  space  of  a  shared  (common)  memory 
is  distributed  among  each  applications 
processor.  Data  and  parameters  to  be 
passed  to  various  process  )rs  from  er 
processors  are  broadcast  globally  on  the 
system  data  bus  but  are  read  locally  in 
each  processor  in  a  parallel  manner.  This 


significantly  reduces  the  system  data  bus 
bandwidth  requirements  and  the  likelihood 
of  bus  contention. 

c.  The  processing  frame  scheduling  (unique  to 
real-time  sampled  data  processing)  is 
carried  out  by  the  control  processor. 
Groups  of  microcomputers  are  scheduled 
according  to  the  processing  required  in  a 
given  frame  time.  All  interprocessor 
communications  are  controlled  by  the  ATM 
which  iz  stored  in  a  PROM. 

The  microcomputer  configuration  is  interesting  for 
future  applications,  but  the  research  program  tas 
not  yet  provided  the  answers  needed  to  use  this 
approach  on  a  simulator  procurement.  This 
configuration  is  included  in  the  evaluation  with 
the  high  risk  of  such  an  approach  made  part  of  the 
analysis. 

EVALUATION  OF  CONFIGURATIONS 
Availability 

The  inherent  availability  of  a  system  is  the 
probability  that  the  system,  when  properly  used  and 
adequately  maintained,  will  operate  satisfactorily 
at  any  point  in  time.  This  definition  excludes 
preventative  maintenance  actions,  logistics  supply 
time  and  administrative  downtime.  The  inherent 
availability  is  expressed  as: 

MTBF 

A  - - 

i  MTBF  ♦  MTTR 

where  MTBF  is  the  mean  time  between  failures  for 
the  system  considered,  and  MTTR  is  the  mean  time  to 
repair  a  failure. 

Availability  of  the  simulators  for  training  is 
a  major  consideration  in  the  system  design.  For 
this  analysis  each  simulator  is  assumed  to  consist 
of  a  cockpit,  an  instructor  station  ana  that 
portion  of  the  total  computer  system  required  to 
make  it  work.  The  reliability  and  maintainability 
numbers  used  in  this  analysis  are  given  in  Table  H. 
The  inherent  availability  is  shown  in  Table  5. 

Maintenance 

Maintenance  of  the  computer  system  includes 
both  hardware  and  software.  At  the  level  of  this 
analysis,  the  distinction  between  contract  and 
in-house  support  in  either  area  cannot  be  made. 
Such  a  determination  will  require  further  study 
after  a  computer  configuration  is  determinei.  A 
ha  *oware  cost  estimate  of  one  per  cent  of‘  the 
initial  cost  of  the  equipment  per  month  is  used  for 
this  analysis.  This  estimate  is  derived  from  a 
study  of  maintenance  contract  charges  for  several 
vendors . 

An  exception  is  made  in  the  case  of  the 
microprocessor  system,  for  which  vendor  maintenance 
is  not  available.  The  cost  of  maintenance  for  the 
microcomputer  system  (except  for  the  peripherals) 
is  estimated  at  two  per  cent  of  the  hardware  cost. 
Peripheral  maintenance  is  estimated  at  one  per 
cent,  just  as  in  the  other  configurations. 

The  cc3t  of  this  software  support  is  estimated 
to  be  ten  per  cent  of  the  initial  software  cost  per 
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TABLE  k.  RELIABILITY  AND  MAINTAINABILITY  VALUES  USED  IN  ANALYSIS 
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year.  This  estimate  is  based  loosely  on  the  work 
of  Putnam  and  Wolverton  [11],  which  finds  that 
support  costs  are  150  per  cent  of  the  acquisition 
cost  for  most  software  systems.  For  the  simulator 
application,  which  should  require  less  support  than 
a  data  processing  application,  this  cost  factor  is 
estimated  to  be  less  than  the  normal  amount. 
Therefore,  the  software  estimate  used  in  this 
analysi"  is  100  per  cent  of  the  acquisition  cost, 
or  ten  per  cent  of  the  acquisition  cost  per  year. 


computers  required.  Those  configurations  that  use 
multiple  computers  to  drive  multiple  cockpits  are 
much  less  sensitive  to  increases  in  requirements. 
In  a  multiple  computer  configuration,  the  entire 
spare  capacity  of  the  computer  system  can  be  used 
for  a  single  change.  If  a  change  does  require 
addition  of  a  computer,  the  entire  spare  capacity 
added  is  available  to  all  simulators  as  needed. 

An  increase  in  the  number  of  cockpits  is  a 
simple  change  for  the  configuration  shown  in  Figure 
2.  An  added  cockpit  requires  the  addition  of  a  new 
cockpit  and  an  associated  computer,  an  addition 
that  is  totally  independent  of  the  existing  system. 
Addition  of  another  cockpit  is  the  same.  For 
multiple  computer  configurations,  the  addition  of  a 
cockpit  can  be  achieved  by  adding  another  computer 
to  the  network  and  making  the  appropriate 
programming  changes.  The  major  difference  is  that 
the  total  system  software  must  change  and  that  some 
lost  time  will  be  encountered  in  introducing  a  new 
system  on  a  shared  computer  complex. 

Large  Computer ♦  For  the  system  using  a  single 
larga  computer,  any  addition  up  to  the  capacity  of 
the  computer  is  a  relatively  simple  change.  Since 
all  spare  capacity  resides  in  one  computer,  a 
change  may  make  use  of  the  entire  spare  capability. 
When  an  addition  exceeds  the  capacity  of  the 
computer,  a  serious  problem  is  encountered.  A 
choice  must  be  made  to  add  a  second  very  expensive 
large  computer  or  to  add  a  stand-alone 
minicomputer.  The  use  of  a  minicomputer  may  reduce 
the  cost  of  the  addition,  but  it  adds  to  the 
maintenance  problem  by  irtroducing  a  different  type 
of  computer  with  associated  problems  in  spare 
parts  and  in  training  requirements.  It  complicates 
the  software  maintenance  problem,  even  though  the 
computer  program  is  written  in  a  high-level 
language.  The  software  personnel  must  learn  a  new 
operating  system  and  I/O  handlers  and  the 
idiosyncrasies  of  the  high-level  language  compiler 
for  the  new  machine. 

Super-Minicomputer .  The  expandability  for  the 
super-minicomputer  configuration  is  very  much  like 
the  situation  with  a  large  computer.  A  single 
computer  handles  several  cockpits,  making  an 
addition  in  the  computation  load  up  to  half  the 
spare  capacity  of  the  system  relatively  easy. 
However,  any  expansion  greater  than  that  requires 
addition  of  new,  relatively  large  computer. 

Microprocessor .  The  microcomputer 
implementation  offers  the  easiest  expansion.  The 
modular  nature  of  the  system  allows  ny  change  to 
be  made  with  a  minimum  of  additional  computers. 


Expandabii ity 

Expandability  will  he  considered  in  three 
parts:  (1)  an  increase  in  the  capability  of  each 
individual  simulator,  (2)  the  addition  of  one 
complete  cockpit  to  the  system,  and  (!)  the 
addition  of  two  cockpits  to  the  system. 

Minicomputers.  A  reasonable  increase  in 
capability  for  each  cockpit  requires  no  change  in 
the  minicomputer  implementations  considered, 
because  a  sp„re  capability  is  built  into  the 
computer  requirements.  An  increase  that  exceeds 
the  spare  capability  provided  presents  a  severe 
problem.  In  the  implementation  shown  in  Hgure  2, 
such  an  increase  can  lead  to  doubling  the  number  of 


LI fe  Cycle  CosL 

The  life  cycle  cost  model  considers  only  those 
elements  associated  witn  the  computer 
configuration.  The  items  used  in  the  simplified 
life  cycle  cost  model  are  derived  below. 

Hardware  Cost  .  Hardware  costs  used  in  the  cost 
model  are  show*.  in  Table  6.  The  computer  prices 
are  based  upon  nominal  prices  for  each  class  of 
computer  with  memory  and  peripherals.  These  costs 
are  based  on  list  prices  contained  in  a  standard 
reference  f  3 1  and  on  information  supplied  by 
various  vendors.  Costs  for  an  actual  system  might 
vary  considerably  because  of  Original  Equipment 
Manufacturer  (OEM)  agreements  that  a  simulator 
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TABLE  h.  HARDWARE  COST  ESTIMATES  USED  IM  ANALYSIS 


:  COMPUTER  TYPE 

COST 
( lOOC’S) 

:  Microcomputer 

4 

:  Minicomputer 

1 40 

:  Super  minicomputer 

3*>o 

:  Large  Computer 

P.3B0 

:  Memory  Bank 

SO 

simulator  vendor  might  have  with  the  computer 
manufacturer.  However,  the  costs  are  considered 
reasonable  for  the  purpose  of  comparing  the 
different  configurations. 

Table  7  shows  the  life  cycle  cost  for  each 
major  item  for  each  configuration  considered.  The 
hardware  cost  is  taken  directly  from  Table  6,  with 
the  following  exceptions: 

a.  Cost  for  a  second  large  computer  is 
reduced  by  20  per  cent,  because  a  complete 
set  of  peripherals  is  not  required. 

b.  Cost  of  a  set  of  peripherals  is  addpd  to 
each  microcomputer  system.  This  cost  is 
estimated  to  be  $40K  per  cockpit. 

Software  Costs .  Software  costs  are  more 
difficult  to  estimate.  The  basic  software  cost  is 
estimated  by  determining  the  program  size  and 
applying  a  cost  factor  based  only  on  the  number  of 
instructions.  Hie  software  cost  for  each  different 
configuration  is  determined  by  modifying  this  basic 
software  cost  to  account  for  any  additional 

TABLE  7.  LIFE  CYCLE  COST  OF  VAR 


j 

complexity  in  the  software  due  to  partitioning  the 
program  to  be  executed  so  that  it  can  be  run  in  a 
multiprocessor  environment. 

The  following  assumptions  are  made  in 
determining  the  basic  software  cost  for  each  type 
of  trainer: 

a.  FORTRAN  is  the  source  language. 

b.  The  FORTRAN  expansion  ratio  is  4  machine 
instructions/FORTRAN  statement. 

c.  Programmers  will  program  at  a  rate  of  2000 
statements/year  including  design,  code, 
and  checkout. 

d.  Labor  costs  are  $90,000  per  man-ye?.r. 

e.  A  CPT  has  a  24K  word  program  or  6K  FORTRAN 
statements . 

f.  An  OFT  has  a  64K  word  program  or  16K 

FORTRAN  statements. 

g.  An  ACMT  has  a  96K  word  program  o~  24K 

FORTRAN  statements. 

The  basic  software  cost  computed  using  these 
assumptions  is  given  in  Table  8.  This  gives  the 
cost  of  programming  for  each  type  of  simulator  when 
the  program  is  executed  entirely  within  one 
computer . 

Many  of  the  configurations  considered  require 
some  partitioning  of  t.;e  problem  to  allow  it  to  be 
solved  in  a  multiprocessor  sycbem.  The  following 
assumptions  are  made  about  the  increase  ir.  software 
complexity  due  to  partitioning. 

[OUS  COMPUTER  CONFIGURATIONS 


CONFIGURATION 

( 

OST  ($1000 

HARDWARE 

ACQUI¬ 

SITION 

SOFTWARE 

ACQUI¬ 

SITION 

ANNUAL 

SOFTWARE 

SUPPORT 

ANNUAL 

HARDWARE 

MAINT. 

10  YEAR 
TOTAL 

3  SYSTEMS 

20  YEAR 
TOTAL 

3  SYSTEMS 

Separate  Computer  Systems 

Each  Cockpit 

1.920 

2,297 

230 

230 

17,266 

26,475 

Separate  Computer  System 

Each  Cockpit  (W/Redun) 

2,060 

2,297 

230 

247 

18, 190 

27,903 

Shared  Computers  with  Min. 
Hardware 

1.370 

2,545 

255 

164 

14, 132 

21,609 

Shared  Computers  with  Min. 
Hardware  (with  redundancy) 

1 , 5 1 0 

2,545 

255 

181 

15,056 

23.307 

Separate  Computers  for  All 

700 

2,070 

207 

84 

8,760 

13.350 

Separate  Computers  fo«-  All 
(with  redundancy) 

1,050 

2.070 

207 

126 

11,070 

16,920 

Single  Large  Computer 

2,  300 

2,070 

207 

276 

19.320 

29.670 

Single  Large  Computer 
(with  redundancy) 

4,  140 

2,070 

207 

497 

31,464 

43.438 

Microcomputers 

592 

3.737 

374 

89 

11,928 

18, 344 
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TA3LE  8.  SOFTWARE  COST  ESTIMATES  USED  IN  ANALYSIS 


TABLE  9.  SOFTWARE  COST  FOR  VARIOUS  CONFIGURATIONS 


TYPE 

EFFORT 

COST 

(1000'S) 

CPT 

3  MY 

270 

OFT 

8  MY 

720 

ACMT 

12  MY 

1080 

a.  The  program  size  increases  by  ten  percent 
for  each  separate  computer  system  over 
which  the  simulator  software  is 
distributed . 

b.  The  software  development  effort  for  a 
partitioned  program  grows  as  the  square  of 
the  increase  in  program  size. 

The  first  assumption  is  an  estimate  of  the  growth 
in  program  size  due  to  increased  need  for  control 
and  communication  between  program  modules.  The 
second  is  based  upon  the  added  complexity  of  the 
program  because  of  the  partitioning.  The  penalty 
for  partitioning  may  appear  to  conflict  with  the 
advantages  generally  attributed  to  modular 
programming.  However,  reca',l  that  the  partitioning 
is  not  simply  dividing  the  computation  into 

manageable  parts  as  done  in  modular  programming. 
The  partitioning  of  the  simulation  program  to  allow 
sharing  among  several  computers  requires  concurrent 
program  execution  and  the  use  of  shared  storage. 

The  method  used  to  estimate  the  added 
complexity  due  to  partitioning  the  program  is  t? 
assume  tnat  the  number  of  intercommunication 

elements  grows  linearly  with  program  size  and  that 
program  complexity  (and  the  associated  cost  of 
developing  the  program)  is  proportional  to  the 
number  of  intercommunication  paths  available. 

Assume  the  program  without  partitioning  has  some 
number  of  intercommunication  elements  N.  Then  the 
number  of  paths  between  elements  is  N(N-1)/2.  If 
the  program  size,  and  thus  the  number  of  elements 
increases  by  z  factor  of  k  because  of  partitioning, 
the  number  of  paths  becomes  kN(kN-1)/2.  Thus,  the 
number  of  intercommmunication  paths,  and  therefore 
the  complexity,  grows  as  the  square  of  k.  This  is 
the  factor  used  as  a  measure  of  the  programming 
effort  required  when  a  program  is  partitioned  among 
several  processors. 

Table  9  shows  the  number  of  partitions  used  in 
each  configuration  and  the  resulting  software  cost. 
The  partitions  were  determined  by  dividing  the 
computation  problem  among  computers  such  that  the 
computation  rate  limits  were  met  and  the  number  of 
partitions  was  minimized. 

Support  Cost.  Annual  support  costs  are  shown 
in  Table  7.  The  c<*st  for  hardware  support  is 
estimated  as  one  per  cent  of  the  initial  hardware 
cost  per  month  per  system.  The  annual  cost  for 
software  support  is  estimated  as  ten  per  cent  of 
the  initial  software  cost.  The  reason  for  these 
estimates  was  explained  above  in  the  discussion  of 
ma'ntenance. 


CONFIGURATION 

SOFTWARE  PART 

rioNS 

SOFTWARE  COST 
$1000) 

CPT 

OFT 

ACMT 

Separate  Computer  Systems 

Each  Cockpit 

1 

1 

? 

?.P4? 

Separate  Computer  System 

Each  Cockpit  (W.'Reiuni 

1 

1 

?<T 

Shared  Computers  with  Min. 
Hardware 

I 

] 

' 

pt 

Shared  Computers  with  Min. 
Hardware  (with  recundancy' 

1 

T 

Super  M.ni-Computers 

1 

1 

1 

?. 

•Super  Mini -Computers 
with  redundancy 

1 

1 

?.  c’o 

Single  Large  Computer 

1 

1 

070 

Single  Large  Computer 
■with  redundancy! 

1 

1 

p 

Microcomputers 

6 

'V 

Risk 

Risk  in  the  development  of  the  software  is  an 
important  factor  in  the  choice  of  an  architecture 
for  a  simulator  complex.  The  single  large  computer 
presents  the  lowest  risk  (provided  the  initial 
estimate  of  the  computer  requirement  is  not  so  low 
that  it  results  in  the  need  for  an  additional 
computer).  The  risk  is  low  because  the  problem 
requires  no  partitioning  and  the  computer  has  the 
highest  flexibility. 

The  single  minicomputer  driving  one  or  more 
cockpits  presents  the  next  lowest  risk.  This 
implementation  does  not  have  the  flexibility  of  the 
large  single  computer,  but  it  requires  no 
partitioning  of  the  problem  and  does  not  require 
several  computers  to  work  together.  Multiple 
computers  performing  the  job  result  in  the  highest 
risk.  The  increased  risk  in  a  multicomputer 
implementation  of  the  problem  is  the  result  of  the 
need  for  partitioning  the  program  among  the 
computers  and  the  concurrent  operation  of  the 
programs  with  its  associated  timing  and 
synchronization  problems.  Thus  the  configurations 
shown  in  Figures  2,  3  and  5  present  successively 
higher  risk  factors. 

The  microcomputer  implementation  provides  the 
highest  risk,  so  high  that  such  an  implementation 
is  not  recommended.  Neither  the  partitioning 
problem  nor  the  control  problem  has  been 
solved — this  alone  is  enough  to  eliminate  the 
microcomputer  from  consideration.  In  addition,  the 
level  of  support  software  available  for 
microprocessors  is  far  below  what  is  available  for 
minicomputers  or  large  computers. 

ADDING  A  REDUNDANT  COMPUTER 

The  computer  complex  is  large  enough  to 
warrant  consideration  of  a  redundant  computer  to 
increase  availability.  Each  of  these 
conf igurations  lends  itself  to  use  of  redundancy  in 
order  to  improve  system  availabil  it’-.  Tt  is 
assumed  in  this  analysis  that  the  additional 
computers  do  not  interact  and  that  the  device  for 
switching  computers  in  and  out  of  the  circuit  does 
not  affect  the  reliability.  Although  these 
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assumptions  cannot  be  realized  in  practice,  the 
effect  is  minor  and  does  not  change  the  result  of 
the  analysis. 

The  determination  of  whether  a  redundant 
computer  is  cost  effective  requires  the  calculation 
of  the  increased  availability  the  results  from  the 
addition  and  the  derivation  of  costs  for  simulator 
downtime.  Assume  the  simulator  system  costs  shown 
in  Table  10.  The  total  acquisition  cost  for  three 
simulator  complexes  is  then  $°0  million.  IC  the 
life  cycle  maintenance  cost  is  assumed  to  be  one 
per  cent  of  the  initial  acquisition  cost  per  month 
as  it  was  in  the  computer  system  analysis  (a  very 
conservative  estimate  when  the  whole  complex  is 
considered),  the  life  cycle  cost  for  the  complexes 
is  $198  million.  Since  this  is  an  order  of 
magnitude  greater  than  the  life  cycle  cost  of  the 
computer  configurations  being  considered,  a  single 
cost  for  an  hour  of  lost  time  applicable  to  any  of 
the  configurations  can  be  established. 


TUPLE  Hi.  OVT  "F  >'WN7Hf 


TYPE 

PORT 
(  1000  *S 1 

HARDWARE 
'PST  PFR 

MOHR 

LAF^R 
"OF  T  PF.R 

FOUR 

'  TAL 

PER 

HOUR 

OPT 

ROO 

•r 

HO 

OFT 

100 

AR 

V' 

wo/vrsiJAi.  , 

OFT 

.■(000 

;> 

Hr 

-'r 

W  'VISUAL 

A"MT 

00  00 

•'IV 

Tf  the  simulators  are  used  52  weeks  per  year, 
6  days  per  week,  12  hours  per  day  over  a  10  year 
life,  one  hour  of  time  on  the  simulators  based  on 
first  cost  is  the  amount  shown  in  the  third  column 
of  Table  10,  The  cost  of  the  lost  personnel  time 
should  al3o  be  included.  On  the  average,  each 
cockpit  will  have  one  pilot  and  one  instructor.  If 
manpower  costs  are  assumed  to  be  $20  per  hour,  this 
gives  the  personnel  cost  shown  in  the  fourth  column 
of  Table  10.  Tne  fifth  column  shows  the  total  cost 
per  hour  of  downtime  for  each  type  of  simulator. 

The  lost  time  caused  by  computer  failure  for 
each  system  is  calculated  from  the  data  used  in 
computing  the  availability.  Column  two  of  Table  11 
shows  the  result  of  applying  the  lost  time  to  the 
cost  data  given  in  Table  10.  This  is  the  cost  of 
lost  time  for  three  configurations  of  computers.. 
Table  11  shows  the  saving  achieved  by  adding  a 
redundant  computer  to  four  of  the  configurations. 
No  redundancy  is  considered  in  the  microcomputer 
implementation,  because  its  one-card  configuration 
allows  changing  of  the  whole  computer  as  the  normal 
mode  of  maintenance. 

Column  four  of  Table  11  shows  the  life  cycle 
cost  of  adding  a  redundant  computer  to  each  system 
configuration.  The  saving  achieved  by  adding 
redundancy  is  greatest  for  the  minicomputer  systems 
and  not  as  great  for  the  super  min. computer 
systems.  In  the  case  of  the  large  computer,  the 
cost  of  redundancy  exceeds  the  cost  of  the  downtime 
that  can  be  saved. 
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RELATIVE  ADVANTAGES  AND  DISADVANTAGES 

The  relative  rutings  of  each  configuration 
considered  in  several  important  areas  are  given  in 
Table  12.  Availability  and  cost  comparisons  are 
based  upon  data  shown  in  Tables  5  and  7.  The 
assessment  of  expansion  capability  and  risk  is 
based  upon  the  discussion  given  above. 

RECOMMENDED  SYSTEM 

The  recommended  configuration  is  the  one  using 
super-fast  minicomputers.  Thre .  computers  are 
required,  two  to  handle  the  computation  load  and  a 
spare  computer  to  increase  system  availability. 
This  syste.,j  has  a  cost  advantage  over  a  system 
using  conventional  minicomputers  and  provides  a 
reasonable  expansion  capability.  It  is  not  as 
flexible  in  adding  another  cockpit  to  the  system, 
because  any  addition  beyond  the  capability  of  the 
two  active  computers  proposed  requires  addition  of 
a  relatively  expensive  computer. 

SUMMARY  AND  CONCLUSIONS 

This  study  provides  an  evaluation  of  computer 
configurations  that  might  be  applied  to  a  simulator 
complex.  The  computer  configurations  that  have 
been  considered  give  an  insight  into  the  effect  of 
various  computer  arrangements  in  terms  of  cost, 
maintenance,  availability,  and  risk.  More 
important  than  the  numbers  themselves  is  the 
procedure  used,  because  this  procedure  can  be  used 
to  address  other  configurations  and  other  numerical 
values  for  the  variable  involved.  This  procedure 
is  recommended  for  use  in  evaluating  more  detailed 
implementations  provided  by  contractors. 

The  treatment  of  computers  as  limited  to 
consideration  of  the  general  characteristics  of 
each  generic  class  of  computer,  with  no  assessment 
of  specific  manufacturer's  products.  The  classes 
considered  are  (1)  large  geeral -purpose  computers, 
(2)  super  minicomputers,  (3)  minicomputers  and  (A) 
microcomputers.  Mine  configurations  were 
considered,  two  using  large  computers,  two  using 
super  min lcomputers ,  four  using  minicomputers,  and 
one  using  microcomputers.  A  summary  of  the 
tradeoff  among  the  various  systems  shows  the 
relative  advantage  of  each  approach. 
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TrtBLE  12.  TRADEOFF  SUMMARY  TABLE 


:  DESCRIPTION 

MINI-COMPUTER  SYSTEM  : 

SEPARATE 

EACH 

COCKPIT 

SEPARATE 

EACH 

COCKPIT 

W/REDUN 

SHARED 

MIN. 

HDWR 

SHARED 

MIN. 

HJWP 

W/REDUN 

SUPER 

MINI 

COMPUTER 

SUPER 

MINI 

COMPUTER 

W/REDUN 

SINGLE 

LARGE 

COMPUTER 

SINGLE 

LARGE 

COMPUTER 

W  RE  DUN 

MICRO¬ 

COMPUTER 

SYSTEM 

:  HARDWARE  COST 

MEDIUM 

HIGH 

MEDIUM 

HIGH 

MEDIUM 

MEDIUM 

LOW 

LOW 

HIGH 

VERY 

HIGH 

LOWEST 

:  SOFTWARE  COST 

MEDIUM 

MEDIUM 

MEDIUM 

MEDIUM 

LOW 

LOW 

LOW 

LOW 

HIGHEST 

:  :  MAJOR 

•  ~XPAN^InN  * 

LOOP 

GOOD 

GOOD 

GOOD 

MEDIUM 

MEDIUM 

POOR 

POOR 

GOOD 

:  :  MINOR 

FAIR 

FAIR 

FAIR 

FAIR 

FAIR 

GOOD 

POOR 

GOOD 

GOOD 

:  SYSTEM  AVAILABILITY 

FAIR 

GOOD 

FAIR 

GOOD 

FAIR 

GOOD 

POOR 

GOOD 

GOOD 

:  MAINTENANCE/LOGISTIC  COST 

MEDIUM 

HIGH 

MEDIUM 

HIGH 

MEDIUM 

MEDIUM 

LOW 

LOW 

HIGH 

VERY 

HIGH 

LOWEST 

:  LIFE  CYCLE  COST 

MFDIIJM 

HI  1H 

MEDIUM 

HIGH 

MEDIUM 

MEDIUM 

MEDIUM 

LOW 

LOW 

MEDIUM 

HIGH 

VERY 

HIGH 

LOWEST 

:  RISK 

LOW 

LOW 

LOW 

LOW 

LOW 

LOW 

LOW 

LOW 

HIGH 

Recommended  System 

The  analysis  indicates  that  the  best  approach 
to  implementation  of  the  simulators  is  use  of  super 
minicomputers.  This  type  of  computer,  now  becoming 
available,  offers  a  significant  cost  advantage  over 
the  minicomputers  used  in  the  most  recent 
simulators.  The  large  computer  and  the 
microcomputer  were  found  not  suitable  for  this 
appl ication . 

This  is  not  a  surprising  result.  It  affirms 
the  fact  that  the  multiple-cockpit  flight  simulator 
is  not  significantly  different  from  flight 
simulators  of  recent  vintage  and  should  be 
implemented  in  much  the  same  fashion.  The 
difference  in  the  recommended  system  for  this 
application  is  the  recent  development  of  super 
minicomputers  which  provide  several  times  the 
capability  of  the  minicomputers  of  a  few  years  ago. 
The  recommended  configuration  consists  of  a  super 
minicomputer  performing  the  computations  required 
for  a  CPT,  four  OFT's  and  an  ACMT.  Three  computers 
are  ir.cluded  ;n  each  complex,  two  computers  to 
share  the  computation  load  and  a  redundant  computer 
that  can  be  used  to  replace  either  of  the  others  in 
case  of  failure.  This  configuration  provides  a 
significant  cost  advantage  over  a  system  using 
conventional  m' nicompute'-s . 

The  use  of  a  large-scale,  general-purpose 
computer  is  rejected,  because  that  approach  is  more 
expensive  than  use  of  multiple  minicomputers  and 
results  in  a  lower  availability.  Although  the 
lower  availability  could  be  corrected  by  the  use  of 
a  redundant  system,  the  addition  of  a  second 
computer  makes  the  cost  tradeoff  even  less 
attractive.  It  is  instructive  to  look  at  the 


reason  for  this  phenomenon  in  view  of  the  use  of 
large  computer  complexes  for  ticket  reservation 
systems  and  other  applications  that  seem  to  require 
much  processing  and  the  same  high  degree  of 
reliability.  The  difference  is  that  the 
applications  where  the  large  computers  show  an 
advantage  are  those  where  the  p-oblem  is  the 
processing  of  vas.  amounts  of  related  data.  The 
simulator  application  is  a  problem  where  a  dozen 
entirely  separate  problems  are  being  solved. 

If  large  computers  are  not  the  direction  to 
go,  it  would  seem  that  smaller  computers  must  be 
the  right  answer.  In  the  long  range,  this  may  be 
true.  However,  for  immediate  implementation  of  a 
computer  complex,  the  risk  of  entering  a  new 
technology  far  outweighs  the  advantage  to  be 
gained.  The  problems  of  control  of  multiple 
microcomputers  in  the  simulator  environment  is 
being  studied,  but  there  is  no  suitable  proven 
solution.  Partitioning  the  problem  for  th® 
microprocessor  is  another  unanswered  question. 
Software  design  aid3  for  microprocessors  are  not  as 
well  developed  as  those  for  larger  machines,  and 
this  would  add  a  great  risk  to  the  overall  project 
schedule  and  cost  if  microcomputers  were  selected. 
These  problems  should  be  solved  in  the  RAD  arena 
rather  than  on  a  major  simulator  procurement. 

Areas  for  Further  Research 

This  analysis  gives  an  overview  of  some  of  the 
considerations  in  choosing  a  computer  configuration 
for  a  simulator  system.  In  preparing  this  study, 
the  authors  have  been  forced  to  rely  upon  estimates 
based  upon  experience  and  reasonable  extrapolation 
of  knovT.  results.  Vveral  areas  touched  upon  in 
this  stuoy  require  further  research  to  the 
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system  better  and  to  provide  a  firm  basis  for 
choice . 

Super  Minicomputers.  The  new  class  of  super 
minicomputers  appears  to  offer  the  most  attractive 
solutior  to  the  computation  problem.  However,  the 
lack  of  experience  in  applying  this  kind  of 
computer  indicates  a  reed  for  closer  monitoring  of 
contractor  effort  in  evaluating  this  configuration. 

Computer  Benchmarks .  Use  of  the  average 
instruction  execution  time  in  evaluating  computer 
performance  is  one  of  the  major  weaknesses  in  this 
study  and  in  the  trainer  procurement  process.  Use 
of  modern  components  such  as  cache  memory,  pipeline 
processing,  and  floating-point  processors  and  the 
use  of  high-level  languages  provide  computer 

systems  whose  performance  cannot  be  determined  by 
such  a  simple  measure.  Instead  of  measuring 
performance  by  use  of  an  assumed  instruction  mix, 
performance  should  be  measured  by  executing 

computer  programs  typical  of  the  application  on  the 
machine  to  be  evaluated  and  measuring  execution 
time  and  storeige  required.  Developing  and 
validating  such  a  set  of  programs  should  be 
pursued . 

Multiprocessor  Systems.  Use  of  multiple 

processors  for  real-time  simulation  requires 

further  study  to  reduce  the  risks.  Work  in 
partitioning  the  problem,  control  algorithms,  and 
configuration  of  processors  and  memory  for  this 
application  should  be  continued.  This  research 
will  apply  to  both  minicomputer  networks  ir.  the 
immediate  future  and  to  microcomputer  networks  now 
becoming  a  viable  method  for  simulator 

implementation . 

Interrupt-Driven  Systems.  Simulators  for 

training  ordinarily  use  a  fixed  frame  computation 
in  which  the  total  task  to  be  performed  is  divided 
into  subframes  to  be  computed  at  various  rates 
(e.g.,  30,  15,  7.5  times  per  second).  Research 
should  be  performed  to  determine  the  advantages  and 
disadvantages  in  using  in  interrupt-driven  system 
which  allows  subframes  to  be  processed  in  the  time 
left  over  after  the  mainframe  computation.  This 
seems  to  offer  a  more  effective  use  of  time 
available  than  the  conventional  method. 
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Growing  emphasis  on  simulation  of  low  alti*  ‘de  and  air-to-air  tactical  scenarios  has 
greatly  increased  the  requirement  for  simulator  *sual  systems  capable  of  providing  t.he 
pilot  high-fidelity  out-of-the-cockpit  cues.  Evaluation  of  visual  system  performance 
through  simulator  flying  studies  has  been  the  primary  measure  of  system  quality.  However, 
such  studies  can  be  costly  and  time  consuming,  and  often  they  provide  equivocal  results. 

>The  present  study  investigated  the  use  of  psychophysical  measurement  methodology  to  provide 
a  quick,  low-cost  evaluation  of  the  altitude  cues  provided  by  five  visual  system  displays. 

Thirty  Air  Force  pilots  made  estimates  of  the  altitude  above  ground  level  (AGL)  shown 
in  slides  of.  visual  system  displays  varying  in  object  density  and  object  detail.  Slides 
showed  a  90^  field-of-view  scene  taken  in  the  F - 1 6  cockpit  of  the  Advanced  Simulator  for 
Pilot  Training.  Eight  altitudes  (range  50-400  ft  AGL)  were  presented  for  each  visual  scene 
condition.  A  random  sequence  of  40  slides  (8  altitudes  X  5  scenes)  was  presented  three 
times.  Power  functions  relating  perceived  to  actual  altitude  were  determined.  Reliable 
differences  were  found  between  the  displays  which  accorded  well  with  differences  found  in  a 
simulator  flyirg  study  using  the  same  display  environments.  Results  are  discussed  in  terms 
of  display  features  and  the  measurement  methodology.^ 


INTRODUCTION 

As  a  result  of  the  current  trend  in 
flight  simulation  toward  tactical  flight  and 
combat  scenarios,  a  need  exists  for 
methodologies  to  evaluate  the  effectiveness  of 
visual  system  displays  in  providing 
out-of-the-cockpit  flight  cues.  The  simulator 
visual  system  presents  the  pilot  with  a 
variety  of  cues  he  needs  to  perform  his  task. 
These  range  from  airspeed,  altitude,  and 
navigation  cues  to  cues  relating  to  the 
presence,  range,  and  behavior  of  threats  and 
targets.  Simulator  flying  studies  have  been 
performed  to  determine  the  effectiveness  of 
texture  (Edwards  et  al,  1Q81),  color  (Kellogg 
et  al,  1981),  and  three-dimensional  objects 
(Rinalducci  et  al,  1981)  in  providing 
low-altitude  flight  cues.  While  such  studies 
provide  the  ultimate  measure  of  the 
effectiveness  of  a  visual  system  disolay  in 
providing  cues  needed  to  perform  simulated 
flight  tasks,  they  can  have  severe 
methodological  limitations.  The  requirements 
of  such  studies  for  simulator  time,  subject 
time,  and  development  time  are  great.  Simply 
to  study  the  effectiveness  of  one  type  of 
visual  cue  can  require  as  much  as  50  hours  of 
simulator  time,  even  if  only  a  small  number  of 
subjects  is  run.  Therefore,  only  a  limited 
n jivber  of  visual  environment  displays  may  be 
investigated.  In  order  to  perform  the 
parametric  studies  required  for  the  desiqn  of 
effective  simu’ator  visual  environments, 
techniques  are  required  for  assessing  the 
cueing  effectiveness  of  visual  displays 
quickly  and  at  low  cost.  Such  techniques 
might  he  used  to  screen  candidate  displays  so 
that  only  the  most  effective  need  be  examined 
in  more  comprehensive  simulator  flight 
studies. The  purpose  of  the  present  research 
was  two-fold.  The  primary  purpose  was  to 
determine  th  effectiveness  of  a  methodology 
for  assessinq  the  quality  of  simulator  visual 
displavs  quickly  and  at  low  cost.  The 
technique  investigated  involved  having  pilots 
estimate  tne  altitude  above  ground  level  ( AGL  1 
in  static  (slide)  presentations  of  a  simulator 


visual  system  oi splay.  This  technique  permits 
"mass  production"  evaluation  of  visual 
displays  but  does  not  permit  examination  or 
evaluation  of  the  capability  of  visual  scenes 
to  provide  altitude  cues  based  on  scene 
dynamics.  Because  of  this  limitation  in  the 
quality  of  altitude  cues  available,  it  was 
necessary  to  compare  the  results  of  tue  static 
estimation  technique  with  those  of  a  dynamic, 
simulator  flying  approach. 

A  second  purpose  of  the  research  was  to 
examine  the  aspects  of  scene  content,  object 
density,  and  object  detail  in  order  to 
determine  their  effects  on  the  perception  of 
altitude.  Two  aspects  of  visual  scene  content 
were  investigated.  These  were  the  density  of 
3-dimensional  objects  in  the  environment  and 
the  level  of  detail  of  the  objects. 

METHOD 

Materials 

Stimulus  materials  were  35  mm  color 
slides  taken  with  a  90°  f ield-of-view  lens 
in  the  F-16  cockpit  of  the  Advanced  Simulator 
for  Pilot  Training  (ASPT)  at  Williams  AFB. 

The  out-of-the-cockpit  scene  consisted  of  flat 
terrain  with  POO  ft  aiming  towers  at 
eight -mile  separation.  Altitude  cues  of 
varying  quality  were  provided  by  inverted 
pyramids.  Condtion  1  and  condition  2  were 
high  detail  conditions  in  which  the  sides  of 
the  pyramids  were  black  and  the  base  (top)  was 
white.  Condition  3  and  condition  4  were 
medium  detail  conditions  in  which  the  pyramids 
were  all  black.  Two  conditions  (conditions  1 
and  3)  we*-e  high  object  density  conditions, 
with  the  mean  distance  between  pyramids  equal 
to  1500  ft.  In  the  low  density  conditions 
(condit itons  ?  and  4),  separation  between 
pyramids  was  4500  ft.  In  all  four  conditions 
the  pyramids  were  50  ft  tall.  A  fifth 
condition  was  intended  to  have  lowest  detail. 

In  this  condition,  the  pyramids  were  displayed 
so  that  only  the  base  was  visible  above  the 
ground.  Thus,  the  pyramid  had  the  appearance 
of  a  triangle  laying  flat  on  the  ground. 
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Unfortunately,  the  level  of  detail  was  so  low 
that  the  triangles  appeared  only  as 
scintillations  in  the  dynamic  scene  as  the 
objects  moved  across  the  raster  lines  of  the 
CRT  display  and  were  nearly  invisible  in  the 
static  display.  As  a  result,  only  the  200  ft 
aiming  towers,  separated  by  eight  miles, 
provide  any  real  altitude  cue.  In  all 
conditions,  eight  altitudes,  ranging  from  50 
to  400  ft  in  50  ft  increments,  were 
presented.  A  single  set  of  40  slides  was  used 
in  which  each  of  the  40  alt -tude-visual 
environment  conditions  was  presented  once  in 
random  sequence. 


Subjects 


Thirty  pilots  in  A-10  combat  crew 
training  served  as  subjects.  Their  -flying 
experience  ranged  from  approximately  400  hr  to 
3000  hr.  None  had  had  any  previous  experience 
in  ASPT. 


Procedure 

Subjects  were  run  in  groups  of  ten  in  a 
squadron  meeting  room  having  projection 
facilities.  Subjects  were  seated  from  15  to 
25  feet  from  the  screen  image,  which  was  7-1/2 
ft  wide.  At  the  start  of  the  session,  the 
experimenter  explained  the  purpose  of  the 
research.  The  experimenter  then  explained 
that  a  sequence  of  40  color  slides  showing 
straight  and  level  flight  would  be  presented. 
Subjects  were  told  that  the  slides  would  show 
five  different  simulator  visual  environments. 
Since  none  of  the  subjects  had  ASPT 
experience,  they  were  told  that  the  range  of 
altitudes  would  be  50  to  400  ft.  Subjects 
were  not  informed  of  the  size  of  the  inverted 
pyramids  since  no  attempt  was  made  to  equate 
the  size  of  the  static  display  with  that  of 
the  display  in  the  simulator  cockpit. 

Subjects  were  then  given  response  sheets  and 
told  that  when  the  first  slide  appeared,  they 
were  to  estimate  the  altitude  (AGL)  shown. 
Estimates  for  subsequent  slides  were  to  be 
made  relative  to  the  first.  Thus,  if  the 
estimated  altitude  for  the  first  slide  was  100 
ft  AGL  and  the  second  slide  appeared  at  twice 
as  high  an  altitude,  the  estimate  would  be  200 
ft  AGL.  Subjects  viewed  the  sequence  of  40 
slides  three  times  without  feedback.  Each 
slide  was  presented  for  eight  seconds  with  the 
interval  between  slides  being  only  the  cycle 
tvne  of  the  carrousel  projector. 


RESULTS 


The  first  slide  presentation  sequence 
was  treated  as  practice  and  tne  altitude 
e'limates  from  the  second  and  third  runs  only 
were  analyzed.  The  data  were  analyzed  by 
first  transforming  the  actual  and  estimated 
altitudes  to  logarithms.  Estimated  altitudes 
were  then  adjusted  for  each  subject's  overall 
bias.  A  least-squares  linear  function  was 
determined  relating  log  estimated  altitude  to 
log  actual  altitude  for  each  of  the  five 
visual  scenes.  The  slope  of  the  log-log 
1  incur  function  is  the  power  of  the  function 
relating  estimated  altitude  to  actual 


altitude.  A  power  (slope)  of  1.0  indicates 
perfectly  accurate  estimate  of  altitude.  A 
power  of  greater  than  1.0  indicates  expansion 
or  overestimation  of  changes  in  altitude,  and 
a  power  of  less  than  1.0  indicates  compression 
or  underestimation  o‘‘  changes  in  altitude 
(Kling  and  Riggs,  19’’ I ) . 

In  addition  to  giving  *he  relationship 
between  actual  and  estimated  changes  in 
aUitude,  the  technique  gives  a  zero 
intercept.  This  zero  intercept  can  be 
considered  an  estimate  of  what  it  would  look 
like  "sittiny  cn  the  deck"  under  the  display 
conditions  used  in  the  study,  although  it  may 
not  be  an  accurate  estimate  of  the  perceived 
zero  in  the  simulator.  However,  in  a  relative 
sense,  the  zero  measures  one  aspect  of  the 
quality  of  altitude  cues  provided  by  the 
different  visual  scenes. 

Altitude  estimates  from  all  five  visual 
scenes  showed  marked  compression  (see  Table 
1).  In  addition,  intercepts  for  all  scenes 
were  inflated.  The  inflation  of  the  intercept 
tended  to  be  most  pronounced  when  compression 
was  most  marked.  The  lowest  detail  scene 
(condition  5)  showed  the  greatest  compression 
with  a  log-log  slope  (ji)  of  .197.  The 
intercept.  (A)  was  also  greatest  for  this 
condition  (a  =  1.85).  Thus,  for  this 
condition,  the  estimated  altitude  was  85  ft  at 
50  ft  actual  and  122  ft  at  400  ft 

TABLE  1 


Condition 

Ooject 

Density 

Object 
Detai  1 

B 

A 

1 

High 

High 

.82 

.31 

2 

Low 

High 

.51 

1.1 

3 

High 

Med 

.70 

.51 

4 

Low 

Med 

.50 

1.1 

5 

Low 

Low 

.20 

1.85 

Slopes  (8)  and  Intercepts  (A)  of  the 
Altitude  Tstimation  Function^  for  the 
Five  Visua1  Display  Environment  Conditions 

actual.  Clearly  the  altitude  cues  provided  by 
this  scene  are  very  poor. 

The  best  altitude  cues  were  provided  by 
the  high  detail,  high  density  scene  (condition 
1).  The  power,  8^,  was  .821  and  A  was  .314. 
Thus,  estimated  altitude  was  25  7t  at  50  ft 
actual  and  257  ft  at  400  ft  actual.  Assess¬ 
ment  of  the  differential  aUitude  cueing 
effectiveness  of  the  five  environments  was 
accomplished  by  an  analysis  of  variance  on  the 
slopes  of  the  altitude  estimat:on  functions 
(see  Fable  2).  The  effect  of  object  density 
on  altitude  estimation  was  determined  by 
post-hoc  comparision  of 

TABLE  2 


SOURCE 

MEAN  SQUARE 

D.  F. 

F -RATIO 

TOTAL 

.113 

149 

- 

DISPLAY 

CONDITIONS 

1.684 

4 

39.6 

ERROR 

.043 

116 

_ 

Results  of  Repeated  Measures  ANOVA  on 
Slopes  of  the  Altitude  Estimation  Functions 
for  Five  Visual  Display  Conditions 
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conditions  1  and  3  with  conditions  2  and 
4.  A  Scheffe'  test  showed  the  high  object 
density  conditions  to  provide  significantly 
better  altitude  cueing  than  the  low  object 
density  environments  (F(4,116)  ~  45.5,  tp< 
.011.  The  effect  of  object  detail  was  ’ 
determined  by  comparison  of  conditions  1 
and  2  with  conditions  3  and  4.  This 
differer.ee  was  significant  (F ( 4, 1 1 6 )  = 

10.8,  p  C  .01)  with  the  high  detail 
environments  providing  superior  altitude 
cueing.  There  was  a  suggestion  of  an 
interaction  between  object  density  and 
object  detail.  The  effect  of  object  detail 
on  the  slope  of  the  altitude  estimation 
function,  was  greater  in  the  high  object 
density  condition  than  in  the  low  object 
density  condition.  Unfortunately  the 
detail-density  interaction  failed  to  reach 
significance  (F(4,116)  <<2.4,  .  1 0  >  p  >.05). 

Condition  5  was  ormited  from  the 
post-hoc  analysis  because  it  was  unclear 
how  this  condition  should  be  classified. 
Many  of  the  subjects  reported  that  they  had 
difficulty  in  distinguishing  the  triangles 
from  flaws  in  the  picture.  Tnerefore 
condition  5  could  be  considered  either  i 
low  densitv,  low  detail  condition  or  an 
empty  field  condition,  with  only  a  hori?o 
line  and  the  distant  aiming  towers.  In  any 
event  the  altitude  cueing  effectiveness  of 
condition  5  was  very  poor  indeed  and  was 
substantially  poorer  than  that  of  any  of 
the  environments  containing  3-dimensional 
objects. 

DISCUSSION 

The  differences  obtained  in  subjects' 
ability  to  estimate  altitude  in  the 
different  visual  environments  have  pointed 
up  the  sensitivity  of  the  magnitude 
estimation  technique.  Substantial 
differences  in  altitude  cueing 
effectiveness  were  found  as  a  function  of 
both  object  density  and  object  detail. 

There  was  also  a  suggestion  of  an 
interaction  between  object  oensity  and 
object  detail,  although  this  interaction 
was  oon-sigmf  icant . 

Unfortunatel y  the  present  data  giv° 
no  indication  of  the  validity  of  the 
approach  for  making  judgements  about  the 
effect  of  visual  scene  content  on  simulator 
flying  performance.To  determine  the 
validity  of  the  technique,  it  is  neccessary 
to  compare  the  results  of  the  magnitude 
estimation  technique  with  those  of  a 
simulator  flying  study.  Rinalducci  et  ai 
(198D  performed  a  simulate**  flying  study 
on  the  Advanced  Simulator  for  Pilot 
Training  uS'nq  three  of  the  environments 
used  in  the  present  study  (conditions  1,3 
and  51.  He  monitored  performance  at 
maintaining  a  constant  altitude  of  i  00  ft 
AGl  while  flying  through  each  environment. 
Although  the  variability  in  performance  was 
high,  significant  differences  were  obtained 
between  condition  ’  performance  and 
condition  5  performance,  with  both  average 
altitude  and  RMS  altitude  performance  being 
superior  in  condition  l.  Performance  in 


condition  3  was  slightly  worse  than  that  in 
condition  1  but  was  net  significantly 
different  from  either  condition  1  or 
condition  5.  The  qualitative  similarity 
between  the  present  data  and  those  of 
Rinalducci  et  al  suggests  that  the 
magnitude  estimation  technique  is  sensitive 
to  the  effect  of  altitude  cues  needed  for 
simulator  flight.  I r,  fact  the  present 

technique  may  be  more  sensitive  to  the 
effects  of  visual  display  factors  on 
perception  than  is  simulator  flying 
performance. 

CONCLUSIONS 

1.  The  magnitude  estimation  approach  has 
been  shown  to  be  a  usable  technique  for 
assessing  the  altitude  cueing  effectiveness 
of  visual  displays.  The  technique  is  very 
sensitive  to  differences  in  the  cueing 
effectiveness  of  diferent  visual 
environments.  Results  obtained  with  the 
magnitude  estimation  technique  are 
comparable  to  those  obtained  in  simulated 
flight  studies,  but  the  magnitude 
estimation  technique  provides  equal  or 
greater  sensitivity  at  lower  cost. 

2.  The  density  of  objects  in  the  visual 
environment  was  found  to  be  a  potent, 
determining  facto**  in  the  cueing 
effectiveness  of  visual  displays. 

3.  Object  detail  was  found  to  be  an 
important  altitude  cue.  There  was  a 
suggestion  of  an  interaction  between  object 
density  and  object  deta'l  in  perception  of 
altitude. 
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ABSTRACT 


Ihe  Navy  A  lac t ical  Aircrew  t  omb.n  Training  System  1 1  WIN)  provides  ihc  instrumentation  necessary  to  record 
in-tlijiht  performance  of  aircrews  during  air-to-air  comh.it  training  D.it.i  leeorded  on  I  AIT  N  has  been  .in  import.nit  source 
of  information  tor  the  development  ol  ob|eeti\e  (light  pcrlor'.ii,.iicc  criteria  I’his  p.iper  discusses  research  relate  1  to  the 
development  and  application  ol  in-flight  measures  ol  air  combat  perlormance  Procedures  lor  systematic  development  ol 
aircrew  perlormance  measures  are  identified  and  discussed  \  generic  methodology  is  pioposed  which  will  eventually  lead 
to  a  prescriptive  model  tor  perlormance  measurement  system  development  Some  ot  the  main  applications  ol  objective 
flight  perlormance  criteria  include  training  progress  evaluation,  training  methodology  and  effectiveness  studies,  and  learning 
acquisition  and  transfer  studies 


INTRODUCTION 


Instrumentation 


•  Both  digital  and  graphic  hardcopy  printouts  o!  flight 
data,  aircraft  stale  vectoi  positions,  eockpit  view  ot 
engaged  aiicralt.  and  mission  surumuiy  data 


Ihe  Navv’s  lactical  Aircrew  l  oin b.ti  I  raimiig  Sy stem 
(  I  \l  is,  enables  aircrews  in  monitor  in  real  time  various  ait 
combat  exercise-,  and  through  its  replav  capabiliiy.  provide-  the 
opportunity  to  debrief  and  evaluate  pilot  tactics,  maneuvers,  and 
weapon  delivery  accuracy  I  he  I  S  Air  force  second-generation 
version  of  this  system  is  referred  to  as  Air  Combat  Maneuvering 
Instrumentation  l  AC  Ml)  I  our  major  subsystems  comprise  the 
FACTS  AO  I  hvsti  m 

•  Airborne  ImtrumtnUtion  SubtyHem  (AIS)  -  l  Acs  a  pod 

attached  to  the  aiicralt  which  me.isuies  flight  dynamics 
mlorin.iiion.  senses  weapon  firing  signals,  and  transmits 
data  to  the  ground  through  the  I  IN 

•  Tracking  Instrumentation  Subsystem  (TIS)  -  l  Acs  a 

series  ol  unmanned  remote  tracking  stations  communicating 
wi'ti  a  rn. islet  tracking  station  in  order  i*>  momtoi  aircr.it! 
in  a  spei  ihcd  airsp.ue 


•  C  oinput' ■•i-geneiatc.l  estimates  ol  the  results  ot  weapon 
tiring 


AIS  [ 


Auboiiit  Intl/umrnOtion  SubtyMtm 

•  P.xl  Phyimtly  SitrtiUr  10  AIM  9. 
Ctrord  O"  A  '.ut! 

•  tunifivivon 

A .!  to  Cl l>unj  GiOui'J  t.s  A, i 
Tucking  Inoiumyntjiion  Subtyilrm 

•  iMnn.*  Mrjiu'.'  j  t  qu'tvnfnt 

•  Oiir  SUl.-.m 


•  I*'t«,,fpgj!>  '  He  "  i 

CCS  [  Coni  ml  «nd  Compulation  SubtyHfm 

•  l  .'i«|Kj!r-» 


ODS  ^  D>>pl«y  A  nj  Subtyitrm 

•  llory  (\‘«wVy 

•  a-si  Agil.iiumriiO  OopUO 

•  l.»«  »od  Mrit iy  C4(V*t»l.<y 


•  Conlrol  and  Compulation  Subsystem  (CCS)  l  « inverts 
data  received  from  the  TIT  into  suitable  lomi  lor  display 
I  Acs  a  jvid  attached  to  the  aircr.tli  which  measures  flight 
dvnanucs  intorr'i.rliou  and  senses  weapon  tiring 

•  Display  and  Debriefing  Sub*ys*cm  (DDS)  -  Nerves  as  a 

control  center  and  display  station' 


I  igures  I  and  illustrate  the  major  I  AC  I  N  \l  Ml 
subsystems  and  dietr  interrelationships  Nome  ol  the  more 
important  training  leatures  ol  I  \C|N  \l  Ml  lollow 

•  Real-time  Hocking  including  positron  velocity.  acceleration, 
altitude  and  angular  rale  measurement  ol  .uriralt 
engaged  in  air  combat  naming 

•  Ia|v  playback  ol  flight  history  data,  complete  with 
pictorial  display  ol  the  au-to-atr  engagement  and  voice 
rransmissions 


Figura  1  Tactical  Aircraw  Combat  Training  Syatam 
(TACTS) 
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Figura  2  Gattaral  Configuration  of  TACTS  Subayatama 
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Recent  engineering  advances  on  TACTS  have  incorporated 
insimmcnialion  required  lor  training  aircrews  in  ground  attack 
missions  .is  well  Some  ol  the  added  capabilities  include  No-Drop 
Bomb  Scoitne.  electronic  warlare.  anc'  mine  taxing.  Tins  research 
paper,  however.  will  concentrate  on  the  siaditional  role  ol 
l'AC'TS  At  MI  in  assessing  performance  ol  aircrews  during  air 
combat  training. 

Need  for  a  Performance  Measurement  Sxsiem 


I  he  Na\x  dexeloped  I  AC1S  in  response  to  lessons  learned 
in  V  iel  Nam  indicating  that  C  S  pilot  performance  in  air  combat 
was  below  acceptable  standards  \  plausible  reason  lor  poor 
performance  was  thought  to  be  inadequate  pilot  training  in  the  use 
of  proper  weapon  fiie  boundaries.  It  was  then  suggested  that 
aircrew  training  could  be  substantial!)  improxed  by  proxiding  an 
instrumented  range  able  to  monitor  and  record  air  combat 
engagements  and  to  simulate  weapon  tiring  results.  tACIS  \x.(s 
designed  to  meet  these  urgent  demands  tor  training  instrumentation, 
so  engineering  emphasis  xxas  necessarily  placed  on  real-time 
tracking  of  air-to-air  engagements,  xxeapon  enxelopc  simulation, 
and  enhanced  aircrew  dc'»ruTx\i  the  l)I)S 

P reseii«  range  instrumentation  capabilities  ol  TACTS  provide 
a  amque  opportunity  for  aircrews  to  practice  both  fixing  and 
shooting  tusks  under  realistic  llignl  conditions  and  within  an 
mtoi malion-ric'h  training  environment  It  is  important  to  realize. 
Inxxexer.  that  since  I  \(  IS  is  desu-ned  primarily  to  enhance 
debrieting  of  indixidn.il  an  eombat  engagements,  the  icsulling 
s\  .tern  design  does  not  include  a  capability  to  collect  and  analyze 
cumulative  performance  results  or  to  depict  statistical  trends 
\n alx sis  lit  trend  data  and  application  ol  othei  peiloimaiice 
assess  nent  methods  .ire  needed  lor 


•  I  raining  progress  exaluation  by 

-  operational  aiicrexxs 

-  it. lining  oiftccTs 

•  (.  ombat  ie.idi.icss  estimation  by 

-  Ileel  commanders 

-  l)ol)  .ui.iix sts  groups 

•  1  act ics  dexelopment  bx 

-  Ileel  squadrons 

-  opeiutional  test  and  exaluation  squadintis 

•  Research  I >e\clop  'lent  lest  and  1  valuation  iRDI  \1  )  bx 

-  xxeapon  system  dexelopets 

-  t  loxeimnent  teseatch  ccniets 


Peiloimaiice  assessment  t*  appropn.itelx  relencd  :o  bx 
nistiuciiotial  system  designers  as  the  he*!  im.ni*  to  determine  that 
learning  has  occ'ined  lollowmg  tiaming  Without  peiiormance 
nieasiceiiiciit  thete  can  be  n<-  assui.nice  that  anx  training  sxstem. 
including  I  At  is.  ■  *  meeting  its  vstgn  ob,.ctixes  Application  ol 
peiloini.uice  assessment  pun.  hue*  will  enable  .iiiciew*  undergoing 
naming  oil  I  At  IS  in 

•  I  xaliiate  progiess  toxx.ud  .omplctmg  spec i lie  instate non.t! 
obiectixes 

•  I  stnn.ite  .oml  ai  icadine**  o!  indixnluai  aiiciexx  [.•••nibci* 
and  opeiation.il  units 

•  Diagnose  tfaii'ing  t.ticieiicies  and  pioxide  couectixe 
feedback,  and  in  gene:  >1.  to  lemoxe  ledundancy  and 
mettle teiic  *  in  naming  thuuigb  application  ot  peiloimaiice 
based  leedb.u  k 


Research  Background 

Since  carls  research  related  to  tile  dexelopment  ol  an 
.ombat  performance  measures  xxas  discussed  in  sexeral  technical 
reports  and  previously  published  articles,  it  w:ll  not  be  treated  m 
great  detail  here 

The  lirst  technical  report  xx.n  written  in  "477  tin  the  Nasal 
Aerospace  Research  Laboratory.  and  this  report  described  at 
analytic  fratnexxork  for  conducting  pcrlormance  ciitenon  resc.iich. 
This  analxtic  framework  described  the  I  AC  TS  as  a  system  which 
includes  u.Ts  .*■(■»»'  instrmiury  am  rah  »  cc/hhiy  mission^  and  vpvr'umg 
'•mironnn  w  I  he  technical  approach  used  calls  for  obtaining 
performance  measures  within  a  sxstem  Iramework  xxhich  identities 
the  aboxe  TACTS  elements,  defines  the  training  mission  and  t lie 
operational  eux  ironment.  and  pros  ides  measurement  methods 
sensitise  to  the  influence  ol  variables  identified  in  the  total  I  ACTS 
system  1  ach  set  of  performance  data  collected  on  TACTS  is 
described  and  refeienccd  using  this  sxstem  framework  tie.,  training 
mission,  sxstem  elements  and  opeiatiug  environment!  Ilius 
peiiormance  variations  can  he  related  had.  to  specific  elements  and 
operating  conditions.  It  desirable,  the  combination  of  elements 
(aircraft,  wcipon*.  aircrew*,  etc  '  and  the  training  mission  an'  be 
systematic  •!;>  controlled  to  assess  their  influence  on  sxsie  i 
pcrlormance  4 

A  second  research  report  presented  measurement  criteria 
and  assessment  methods  for  evaluating  Navy  aircrew  miss 
envelope  leeogintion  performance.  Two  generic  measureu 
methods  were  used  to  assess  aircrew  pcrlormance  in  missile  bring 
aceUtacx 

I  ii'sj.  a  ci iterion-relerenced  assessment  method  was  applied 
to  score  missile  launch  success  i percent  liilsi  am!  to  compute  task 
accuracy  measures  (error  trom  prescribed  missile  launch  boundary) 
\  pilot  must  lire  Ins  weapon  within  this  prescribed  boundary  to 
obtain  a  hit.  and  the  dis.aiice  from  a  proper  launch  window  can  be 
used  to  compute  task  accuracy  (c nor  scored)  measures. 

Second,  norm-'  tcrenccd  measures,  based  on  empirical  data 
(mean  r  one  standard  deviation),  were  u*c.t  io  evaluate 
pcrlormance  related  to  group  standards  \  combination  ol  tiorm- 
rcferenced  and  criterion-referenced  nicasine*  has  been  proposed 
lor  other  speettte  an  combat  task*,  t.e.  radar  search  and 
acquisition,  visual  search  and  lajuisitton.  tactics  and  maneuvers, 
etc 

Another  teport  present  ’d  a  synthesis  and  application  ot  the 
above  measurement  methods  In  addition  pieltininary  measurement 
validation  'ests  were  presented  Results  ol  these  prcliminarx 
xalidation  tests  showed  statistical  support  for  selection  ol  ta*k- 
bascd  measures  used  to  asses*  aircrew  perlormance  in  air-to-.m 
combat 

I  mally.  more  extensive  statistical  validation  test*  were 
conducted  using  multiple  lineal  regression  analysis  in  a  do  Me 
cross-validation  design  Measuie*  related  to  successful  completion 
ol  various  an  ionilvt  tasks  have  reliably  piediclcd  4>  percent  ot 
the  x.mancc  in  linal  engagement  outcome  scores  These  tasks 
include  early  visual  i.iemilicaiiou  ot  an  opponent.  missile  the 
acc u lacy,  shootei-to-iargel  pointing  accuiacy.  ami  maneuxciing  to 
obtain  a  fust-shot  advantage 

PERFORMANCE  MEASUREMENT 
SYSTEM  DEVELOPMENT 

Rcse.uch  Strategy 

Meistei  tl‘»'S|  has  summ.it tzed  characteristics  which 
distinguish  measurement  approach-  s  ‘\i*ed  upon  cont-ollcd 
e\peiu”eutation  and  those  nieasutement  appioache*  related  to 
complex  systems  An  important  distinction  made  bx  Mei*tei  <* 
mat  measuic’s  obtained  in  complex  sysic’ms  are  taken  m  reterence 
to  the  entue  ta*k.  oi  |ob.  which  i*  peilouued  in  the  context  ol  the 
actual  tuiKoutiolled)  work  enx  uoiunent 
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Ill  contrast.  laboratory  opcrimcnt.il  research  is  conducted 
under  lughh  controlled  conditions  Isolated  or  synthetic  tasks  are 
peMormed  while  selected  variables  arc  systematically  manipulated 
Measures  obtained  in  the  laboratory  are  usually  directly  u luted  to 
cxpci  internal  hypotheses  which  guide  t'v  measurement  selection 
process,  data  colic  lion,  and  analysis  procedures  used  to  evaluate 
experimental  results  In  unplex  systems.  ineasuics  are  tvpicalh 
obtained  while  the  svst  ,.i  is  m  operation.  I-  is  difficult  to  measure 
the  performance  ol  individuals  under  these  circumstances  because 
each  individual  s  effort  is  embedded  within  a  s\ stem  framework 
Master  ptoposes  to  measure  pc-ioimunce  at  'he  individual,  team 
and  sv stents  levels  1  h is  will  help  us  to  understand  the  relative 
contribution  ol  each  level  in  meeting  overall  system  perlormance 
requirements 

Another  major  diUicultv  encounteied  with,  aircrew 
performance  measure  ivni  in  complex  system-.  is  the  determination 
•>!  •  c i  1  >  what  meusu:  *s  to  i.  e  out  of  the  total  available.  While 

there  aie  manv  studies  of  aircrew  performance  in  the  literature,  no 
widely  accepted  research  strategy  has  evolved  which  specifically 
outlines  the  steps  or  proceduies  required  to  select  and  validate 
aircrew  pei'ormanee  measures  and  assessment  methods. 

\  reuls  and  Wooldridge  ( il>~~)  have  discussed  in  broad 
outline  main  important  aspects  ol  peildrmanee  measurement 
research.  These  authors  purpose  a  sv  stems-oriuiteu.  statistically 
based  approach  to  develop  aircrew  performance  measurement 
methods.  Some  of  the  more  important  ideas  discussed  bv  V reuls 
and  Wooldridge  in  then  article  are  briefly  summ.ir  belov. 


•  Systems  Orientation  -  Aircrew  performance  is  embedded 
in  a  complex  system  and  is  influenced  In  such  factors  as 
tactical  doctrine,  n  »s:« >< »n  plan,  wc.nhet.  weapon 
tvailabilitv.  airer.itl  subsystem  capabilities,  and  operating 
alus 


•  Task  Analysis  -  She  measurement  spcciaii-l  must  devise 
a  means  to  sample  ao.rew  tasks  that  are  most  important 
to  me. nine  ti  e.  lusks  and  measures  that  are  best  for 
describing,  predicting,  and  understanding  aireiew  and 
svsteni  peiforniaiieej 

•  Statistical  Approach  -  \  multi-variate  statistical  approach 
i"  proposed  as  the  onh  method  powerful  enough  to  deal 
with  the  complexity  ol  the  real  world’  <i.e..  in  which 
manv  variable'  interact  to  influence  performance 
measures  obtained). 

•  Measurement  Selection  -  Measures  should  be  selected 
which  satisfy  statistical  requirements  lor  validity  and 
reliability  and  which  are  accepted  bv  the  operational  user 

The  above  researchers  have  .den tilled  some  kev  principles  to 
consider  d’.-ing  'he  development  of  aircrew  performance  ciiiena 
\ppl.C  atiou  ol  sucli  principles  forms  the  basis  oi  a  systems 
approach  to  performance  measurement  specification  which 
emphasizes  generic  methods  and  test  procedures. 

System  Development  Model 


I  igure  ;  presents  a  proposed  model  for  Performance 
Measurement  System  (PMS)  deve!opmen>  1  he  model  specifies  a 
four-phase  research  and  development  program  which  begins  In 
defining  the  mi  dug  mission  and  operational  task  structure  and 
ends  with  implementation  ot  ..  PMS  The  four  phases  of  i>\tx 
development  (analysis,  description,  validation  and  implementation) 
are  briefly  described  below. 

•  Analysis  -  During  the  analysis  phase  of  research, 
information  related  to  the  mission  and  its  associated 
operational  tasks  is  collected.  Operational  task  data  are 
brought  to  a  level  of  analysis  required  to  specify 
performance-based  training  objectives  and  to  m.’cci  a 
pi.Timinarv  measurement  set.  till 


ANALYSIS 


DESCRIPTION 


VALIDATION 


IMPLEMENTATION 


Figure  3  Performance  Measurement  System  (PMS)  Development  Flow  Chart 


385 


•  Description  -  A  \ ;iri;iblc  matrix  which  identifies  training 
or  operational  s\ stem  elements,  in  this  case  aircrews 
aircraft,  weapons  and  mission  factors  The  variable 
matrix  senes  as  an  initial  systems  framework  for  guiding 
performance  data  collection  and  analysis  Data  collected 
on  the  training  system  can  he  categoriz-d  lor  later 
analysis  with  fespect  to  the  key  operating  elements  as 
potential  sources  of  variance  of  the  training  system.  In 
other  words,  statistical  sample  comparisons  are  made 
between  various  aircraft  types.  weapon  types.  mission 
characteristics,  aircrew  changes,  etc.  These  initial 
descriptive  statistical  comparisons  ptovide  the 
measurement  specialist  with  initial  information  regarding 
the  sensitivity  of  measures  to  discriminate  expected 
performance  differences  at  the  system  h  v el.  Table  I 
presents  an  example  of  a  variable  matrix  which  was  used 
in  air  combat  performance  measurement  development. 
(See  Re!  (II).) 


•  Validation  -  Information  I  mm  task  analysis  and 

oe  criptive  statistical  o  npurisons  (i.c..  from  analysis  and 
v.cscriptive  phases)  is  used  as  a  foundation  to  establish  a 
preliminary  measurement  Iramewoik.  The  measuiement 
-framework  represents  a  hypothetical  position  regarding 
operational  task'  their  associated  performance  measures, 
and  the  potential  influence  of  system-level  variables  on 
overall  performance  in  the  opera'tonal  training  system. 
Since  the  initial  measurement  framework  is  based  on  the 
researcher’s  priori  judgments  aho..t  key  v..i  hies  and 
their  relationships  to  the  operating  system,  the  resulting 
framework  repiesents  a  theoretical  structure  requiring 
empirical  validation.  (See  Ref  (4) ) 


The  alidation  phase  of  performance  measurement 
research  is  concerned  with  establishing  an  objective 
rationale  to  verify  ano  support  initi  d  selection  o! 
candidate  performance  measures  I  h*  validation  rationale 
iisiiiilly  is  based  on  the  following  considerations: 

lal  The  degree  to  which  candidate  measures  correlate 
with  overall  (terminal)  system  performance 
<h)  The  ability  of  the  measures  to  discriminate  variations 
in  operator  (aircrew )  skill  level 
<c)  The  reliability  of  performance  measures  obtained 
over  repeated  applications 
(d)  I’scr  acceptability  and  ease  of  implementation 

Once  a  rationale  for  validation  has  been  established  and 
successfully  demonstrated,  then  data  base  requirements 
and  functional  design  features  can  be  specified  for 
hardware  and  software  engineering  development  of  a 
prototype  PMS. 

•  Implementation  -  An  important  part  of  PMS 

implementation  should  be  preparation  ol  .1  lest  and 
I  valuation  ( TA:I.)  plan.  The  LVI  plan  should 

(at  Include  guidelines  for  applying  the  PMS  as  an 
integral  part  of  an  overall  training  system 
lb)  Specify  Training  I  ffeetiveness  I. valuation  (  I  I  I  ) 
procedures  for  PMS  application. 

Procedures  outlined  above  need  to  be  amplified  in  certain 
mens  and  refined  in  others  to  formulate  a  more  complete 
prescriptive  model  for  performance  measurement  system 
development.  I  von  at  this  early  stage  of  their  development,  such 
procedural  gut  lelines  were  a  useful  aid  in  the  specification  of  a 
measurement  Iramewoik  lor  air  combat  performance  assessment 


Table  1.  TACTS  Training  System  Components  and  Variable  Matrix 


Personnel 

Equipment 

Mission 

Instructors* 

Aircrews 

Aircraft 

Weapons 

Mission 

Environment 

Experience 

Adversary 

Adversary 

Type 

Type  (eg  . 

Weather 

and  training 

and  fighter 

and  fighter 

single  or 

aircrews 

type 

Design 

multiple 

Visibility 

In-ftight  and 

characteristics 

aircraft) 

debrief  pro- 

Perceptual/ 

Pertormance 

ana  limitations 

Terrain 

cedures  and 

motor  skills 

characteristics 

Specific 

techniques 

and  limitations 

Delivery 

training 

Vaffic 

Trammg  arid 

parameters 

objectives 

Content  and 

education 

Specific  design 

Sun 

quality  of 

features 

Procedures 

Tactics  and 

directive 

Flight  expert- 

| 

maneuvers 

Miscellaneous 

commentary 

ence  (total. 

Particular 

Specific 

specified 

(related  to 

type,  crew. 

weapon  and 

weapon  toad 

and/or  used 

RTOs) 

section) 

sensor 

and  selection 

complement 

options 

Range 

Training 

ACM 

operating 

function  and 

expeoence 

System  sub 

constraints 

tasks 

system  operating 

status 

03M  inn.it  mp 

Training  aids 

aircraft 

1 _ 

(mission  mix) 

•Including  Range  Training  Officer  (RTOt  and  Flight  Leader 
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Aii  Cooihul  Measurement  lrqmework 

.  igiir.  4  allows  ;i  sim plit'ioci  Air  C’onvkil  Maneuvering 
lAl.'Mi  duee.  ACM  nitwinn  pkust'y  arc  written  at  the  top  of  the 
figure,  while  points!  trf  measurement  related  to  ke>  aircrew  training 
objective*  are  indicated  in  boxes.  The  solid  boxes  represent  points 
lor  which  measures  are  now  mailable,  and  dotted  boxes  represent 
recommended  additions  to  this  o\ erall  measurement  scheme,  lor 
example.  ACM  engagement  state  models  and  algorithms  tor 
measuring  energv  nnmeuverubilitv  performance  are  two  essential 
additions  in  the  final  formation  ot  an  overall  ACM  n  Msurement 
system 


Measures  lor  engagement  state  are  based  on  obtaining  a 
position  ad\antagc.  i.e..  b\  measuring  an  aircraft's  proximitv  to  tfoc 
lethal  /one  of 'an  adversary  aircraft.  Measures  lor  energv 
maneuverability  are  rehitecl  to  optimi/ing  airspeed  for  efficient  aii 
combat  maneuvering,  i.e..  living  an  aircraft  to  its  aerody namic  ideal. 
Addition  of  energy -related  and  m;ineu\ering  information  to  the 
building  blocks  model  presented  in  figure  4  would  complete  a 
technical  approach  that  is  based  on  a  multi-task,  multi-measure 
method.  Table  2  shows  short-form  defnitions  of  ACM  training 
obiccti\es  and  also  indicates  the  corresponding  candidate 
performance  measures  for  these  objectives.  More  detailed 
definitions  lor  training  objectives  and  candidate  measures  are 
reputed  elsewhere.  (See  Kel.  (III.) 


RADAP  SEARCH 
AND 

ACQUISITION 


VISUAL  SEARCH 
AND 

ACQUISITION 


TACTICS  WEAPON  ENVELOPE 
AND  RECOGNITION  ENGAGEMENT 

MANEUVERS  AND  FIRE  OUTCOMES 


L _ J 


V" 


-RANGETIME- 


Figure  4  Simplified  Air  Combat  Sequence:  Shows  Engagement  Phases  and  Measurement  Points 


Table  2.  Air  Combat  Engagement  Analysis 


Training  Objective 


1  Obtain  early  radar  contact  and  lock-on 

2  Determine  adversary  attack  formation  at  10  n  m. 

3  Obtain  early  visual  detection  of  adversary  aircraft 

4  Obtain  early  visua'  identification  of  adversary  aucra't 

5  Determine  attack  formation  at  initial  pass 

6  Maintain  high  energy  state 

7  Gam/maintam  portion  advantage 
S  Ga*n  firing  opportunity 

9  Obtain  first  shot  of  engagement 
10  Fire  weapon  in  weapon  envelope 

1 1.  Obtain  first  kill  of  engagement. 

12  Execute  successful  re-attack 

13  Execute  successful  bugout  by  staying  neutral,  maintaining 
energy,  and  completing  disengagement  with  no  fnendty 
loss 

14  Obtain  favorable  exchange  (Exch)  rale 

15  Satisfy  mission  (utility)  requirements 


Performance  Measure 


Interaircraft  range  and  success  rate  <%>  over  engagements 
ilown 

Quantity  and  position  of  enemy  aircraft 

Interaircraft  range  and  success  rate  (%)  over  engagements 

flown. 

Interaircraft  range  and  success  rate  1%)  over  engagements 
flown 

Quantity  and  position  of  enemy  aircraft 
Indicated  air  speed  3nd  altitude  (energy  package) 

%  or  proportion  of  engagement  in  otfensive.  defensive  stales. 
Time  and/or  %  in  envelope  or  fatal  oftensive  state 
Elapsed  time  and  %  fust  shots 

Irteraircratt  range,  angle-ofl-tail.  pointing  angle,  airspeed  and 
acceleration  parameters 
Elapsed  time  and  %  first  kills 
Iterate  6-1 1  above 

neutral,  indicated  airspeed  and  altitude.  %  loss  at  bugout 

Ratio  ot  fighter  to  adversary  kills 

Neutralize  threat  aircraft  and  survive  or  minimize  losses 
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I'ACTS  h.iining  Application 


Results  of  the  measurement  research  program  ha\e  been 
directly  applied  to  operational  training.  A  computer-based.  air 
combat  Jebriel  system  was  designeil  and  de\eloped  as  a  prototype 
to  test  the  I'easi hi  1  it\  ot'  adding  .1  PMS  to  TACTS  ( 1 4. •  I  > ).  The 
resulting  debriel’  system.  currently  undergoing  further  development 
by  NAYTRAl  Ql'IPC  I  N.  is  referred  to  as  the  Performance 
\ssessment  and  Appraisal  System  (PAAS).  PA  AS  design  is  based 
on  the  previously  described  measurement  framework  which  has 
been  partially  validated.  (See  Ref.  (S).) 

The  debrief  system  can  display  numerous  graphs  depicting 
statistically  summarized  performance  results  from  I  ACTS  training 
sessions.  (See  Ret.  ( l>>.)  Data  are  provided  to  aircrews  m  an 
understandable  format  for  review  and  diagnostic  evaluation  of  air 
combat  training  progress  l  igures  >  and  h  present  examples  of 
compu'er-generated  graphics  used  by  aircrews  to  assess  their 
performance  in  key  aircombai  tasks  using  PAAS. 


•»a«.-#ri«ry  aircraft 
9*tacraan<  3*i«t 


2VJ  Sq-iaaron  vf 

ail.  auv#r»arv  SquaJr^r  •  4L.. 

22Jur<»l-«2’yl81 


RRERR  CONTRC  T 


‘  20  r;  Lata  -  3.0 

L*.  Nona  -  3.0 


•:  Early  -  100.3 


30  !• 


—  Cntonon 
Cutoff 


Figure  5  PAAS  Graphic  Format  Used  to  Evaluate  Radar 
Contact  Performance  for  a  Singie  Training 
Detachment 


it  is  important  to  note  lii.u  the  piesent  P  \  piovules 
peiloimaiKe-basetl  lectlb.it  k  m./i  on  non  in.ni.  tiu  mig  poiliniis  ot 
an  an-to-an  engagement  liulusion  t>t  measures  tel.ited  u*  tu 
voinbut  niaiienveiing  is  t,oiiMtleied  ncvcss.nv  io  complete  the 
pn*pos.it  me.isinenient  tiamcwoik  tpiesented  in  I  igure  -it  Obtaining 
an  1.0m bat  mim>  lot  fine  me.isuies  howevei  will  icijunc  use  ot  an 
automated  data  icincval  'vsiem  that  is  lied  diiatlv  to  I  \CTs  \t  Ml 
in  oidci  to  capture  the  time  hist.ov  data  ncecss.iry  It  i  analv/me 
in.iiieuveis  So  automated  tetneval  svsiem  v  available  * *•* 
t  M  I  s  \C  Ml  todav  and  would  have  to  be  developed 


ifOi. 


r. 

2**  iC'Wc*'4 


H 1 1 a i on  Type 
Rover* ery  rtircreft 

3et*cne#m  Date* 


2V2  Squadron 

AIL  Adversary  Squadron  :  ALL 

22Ju«ll-42Jul*l 


RIM  MISSILE  EIRE  PL  CM 

Range  ( x  l  om,  ) 


\ 

x 


S  4  3  2 _ r 


0  I  *  2  3  4  5 


US  . 


Figure  6  PAAS  Graphic  Format  Used  to  Evaluate  AIM 

Missile  Fire  Accuracy  (envelope  recognition)  for 
a  Single  Training  Detachment 


DISCUSSION  AND  CONCLUSIONS 

Perfoimance  measmement  is  an  npoium  mgicdtcni  to 
effective  naming  It  is  only  bv  measuring  peilormancc  that  we  can 
know  it  learning  has  occuricd.  and  whcthei  ot  not  out  iiistnution 
lias  succeeded 

Until  iccenily.  aiiview  h. lining  insiuieiois  had  to  iclv  upon 
then  subteeiive  estimates  ot  student  naming  piogicss  and 
msiiueiional  effectiveness  Wr.h  the  development  ot  msti umented 
lunges  like  I  \(  IS.  we  can  now  icconl  m-llight  pcibumamc  Such 
a  capability  piovides  the  tiaimng  community  with  an  oppoiiuuiiv 
to  apply  obiccuve  peitormanee  entena  dining  the  instiuctioa.il 
piocess 


(.  aim  .n  should  be  laiseil.  however  com  •■tiling  the  advent  ot 
■  it ^ 1 1 umeiitv. ii  i.mges  like  I  \C  IS  and  advam ed  tligln  sunulatois 
which  iikoipoiate  iceording  capabilities  I  he  mcic  v.ipabihiv  to 
icconl  outputs  liom  these  naming  svsienis  does  not  iu\ess.mlv 
guaiaiitee  that  Mich  iceoided  d.ila  aie  valid  and  tellable  me.isuies 
oi  that  lliev  ,ue  appiopnalelv  applied  \s  should  be  appaieiil  tioin 
the  eoiiienis  ot  this  pa  pci  and  pievious  publuatnnis.  mik  Ii  an.  iIvms 
and  testing  picecdcd  the  selection  and  application  ot  peitonn.nu e 
me.isuies  Sikh  anulvsis  .nut  testing  is  an  essenn.il  picv.iiitioii 
against  meielv  adding  a  PMS  as  .1  possible  evtiau.-oiis  leatuie  In  a 
li. lining  devke  Vililiiion  ot  a  ('M's  van  enhaike  n. lining,  but  it 
also  van  coiivClv ai'lv  iCsiiii  in  1/.1  fnJ*n.’  11. III.  ug  value  d  .«  !' mv>  .. 
|sootlv  designed  ami  01  mappiopnatelv  applied 

Mils  pajvi  vli'viissed  applkation  ot  .1  task-based  mc-isnlctociit 
appio.iv h  wlikli  vapitah/ed  on  die  av.nlabilitv  ot  i  \(  |s  data  l<«i 
•  /« *« /n/'i*ic  ui/ii/.iiifig  ami  .i/y/t mg  an  vombal  jvitoi tnan-.e  nie.i'incs 
I iie  pioposed  I’Ms  development  model  icpiosents  .♦  Impelul  nisi 
step  low.iul  a  mote  letmed  piesciiptive  model  1*1  gotten, 
tncihodologv  !oi  1  iivoipoiat oh1  s\ sienialiv allv  d  -v  eloped  and 
validated  |Vi!oim.nue  .  tilciia  .«s  .01  integral  part  ot  iiaimiig  sssiem 
design 
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EFFECTIVENESS  EVALUATION  FOR  AIR  COMBAT  TRAINING 


JAMES  McGUINNESS ,  Ph.D. 

JOHN  H.  BOUWMAN 
Person-System  Integration 

^  and 

JOSEPH  A.  PUIG 

Naval  Training  Equipment  Center 
ABSTRACT 

This  paper  addresses  methods  developed  for  evaluating  factors  involved  in  Air  Combat  Maneuvering 
(ACM)  training.  In  the  course  of  selecting  and  applying  evaluation  techniques,  a  unique  situation  for 
(  a  transfer-of-training  study  was  presented:  a  newly  installed  ACM  simulator  co-located  with  an  ACM  range. 
A  common,  objective  performance  measurement  system  was  developed  for  the  Air  Combat  Maneuvering  Simula¬ 
tor  (ACMS),  designated  Device  2E6,  and  the  Tactical  Aircrew  Combat  Training  Syotem  (TACTS)  range.  The 
TACTS  range  was  planned  as  the  setting  for  studying  transfer  from  the  simulator  to  an  operational 
situation.^ 


INTRODUCTION 

An  early  attempt  to  evaluate  pilot 
trainees  in  air-to-air  combat  was  known  as 
"pin  ball"  (USAF,  1945).  The  "pin  ball  system" 
registered  hits  of  frangible  bullets  on  a  manned, 
amior-platf'd .  tarqet  aircraft.  The  system  also 
provided  a  visual  signal  to  the  attacking  pilot 
by  turning  on  a  strobe  light  in  the  nose  of  the 
target  aircraft.  The  fidelity,  realism  and 
immediacy  of  objective  feedback  were  high.  As  a 
gunnery  trainer  "pin  ball"  was  a  relatively 
effective  training  device,  even  though  it  probably 
did  little  for  the  pulse  rate  of  the  target 
ai rcrewman. 

In  1967,  NTEC  began  development  of  a 
gunnery  practice  system  based  on  the  use  of  in¬ 
expensive,  eyo-safe  laser  transmitters  and 
receivers  to  simulate  firing  live  rounds.  Laser 
simulation  has  the  advantage  of  Droviding  an  un¬ 
limited  source  of  hazard-free  "u.iinunition"  at 
a  negligible  cost  per  round.  Previous  applica¬ 
tions  of  this  technology  include  the  Laser 
Marksmanship  Rifle  Trainer  and  the  Helicopter 
Door  Gunnery  Trainer. 

The  advent  of  air-to-air  missiles  in  air 
combat  maneuvering  made  the  application  of  PIN 
BALL  training  techniques  very  inefficient,  and 
required  a  technological  sophistication  beyond 
laser  designators. 

Training  Air  Combat  Maneuvering  (ACM)  skills 
requires  the  exercise  of  a  complex  pattern  of 
perceptual,  psychomotor,  physiological  cues  and 
procedural  elements  within  a  demanding  tactical 
environment.  You  cannot  train  such  a  pattern 
in  a  simulator  exclusively.  Neither  can  you 
efficiently  learn  s  ich  a  ;  r  tern  in  an  aircraft 
exclusively.  Training  devices  must  be  design. d 
to  complement  actual  training  in  aircraft.  The 
increased  complexity  ot  tighter  aircraft  include 
multiple  weapons  for  ACM  employment,  multiple 
systems  for  acquisition,  two-man  crews,  addition¬ 
al  functions  required  to  operate  weapons  systems, 
and  increases  in  a;rcraft  performance  which  re¬ 
quire  quicker  aircrew  reaction  time.  All  of 
these  factors  mandate  increases  in  both  flight 
time  and  simulator  usage,  and  effective  ‘ntegra- 
tion  of  training  devices  and  aircraft  into  the 


fleet  ACM  training  program  structure.  Improved 
training  can  have  a  force-multiplier  effect  in 
ACM,  but  we  must  bite  the  bullet.  More  training 
sorties  in  the  air,  undoubtedly,  are  required; 

and  mere  ci  mi  list''"''  hourc  aljn  will  contribute  to 

the  training  process.  Maximum  contribution  of  a 
simulator  is  contingent  upon  the  intenr2tion  of 
proper  maintenance  and  instructor/operator 
support. 

A  rraining  Effectiveness  Evaluation  (TEE) 
was  initiated  by  the  Human  Factors  laboratory  at 
the  Naval  Training  Equipment  Center  (NAVTRAEQUIP- 
CEN)  to  aid  in  defining,  through  objective 
measurement,  how  much  and  whe .  type  of  integration 
is  required.  The  TEE  effort  has  been  designed 
to  provide  data  to  help  determine  how  well  a 
training  system  produces  a  desired  result.  Such 
data  must  be  forthcoming  if  we  are  to  design  for 
(1)  optimal  simulator/aircraft  mix,  (2)  instruc¬ 
tional  strategies  such  as  optimal  sequencing,  or 
(3)  other  significant  factors  that  affect  the 
amount  and  type  of  training  required.  The 
successful  quantification  of  training  effective¬ 
ness  also  wili  provide  a  data  base  which  can  be 
used  for  the  specification  and  development  of 
new  or  modified  simulator  systems. 

The  T Ef  took  advantage  of  a  unique  situation 
for  the  development  of  such  a  data  base.  A 
recently  installed  ACM  simulator  co-located  with 
an  ACM  range  at  LAS  Oceana,  Virginia,  was  reviewed 
for  the  feasibility  of  conducting  a  transfer-of- 
training  study.  A  review  of  current  ACM  aircrew 
training  programs  ,  as  well  as  a  review  of  '"imulator 
and  trailing  devive  applications,  suggested  the 
need  for  a  common  Performance  Measurement  System 
(PMS)  between  the  Air  Combat  Maneuvering  Simulator 
(AChS),  designated  Device  2E6,  and  the  Tactical 
Aircrew  Combat  Training  System  (TACTS)  range. 

Device  2E6  is  a  high  technology  simulator 
consisting  of  two  40- foot  diameter  domes  with  an 
adversary  aircraft  image  projected  on  the  surface 
of  the  dome.  The  adversary  image  moves  in  res¬ 
ponse  to  inputs  from  an  ai rcrewman  in  the  opposing 
dome  or  in  response  to  computer-controlled  or 
con. ole  operator-controlled  inputs.  Simulated 
missile  firing,  as  well  as  gun  firing,  are  possible 
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in  the  simulator  and  are  accompanied  by  visual 
cues.  This  type  of  interaction  can  occur  on  a 
1  versus  1  basis  (Ivl),  on  a  2  aircraft  versus 
1  aircraft  basis  (2vl),  or  on  a  3  aircraft 
against  one  another  basis  (Ivlvl). 

The  TACTS  range  permits  objective  measure¬ 
ments  of  aircraft  spatial  relationships,  missile 
maneuvering  envelopes  and  simulated  missile 
firing  in  an  environment  that  is  as  close  to 
combat  as  is  possible.  The  TACTS  range  instru¬ 
mentation  records  selected  movements  a. id  actions 
in  specially  instrumented  aircraft  as  they  inter¬ 
act  in  a  "dogfight"  over  the  Atlantic  Ocean. 

The  system  tracks  aircraft  operating  within  an 
assigned  airspace  and  computes  and  stores  air¬ 
craft  position,  attitude,  and  weapons-related 
parameters  in  real  time,  permitting  a  three- 
dimensional  depiction  of  the  engagement  to  be 
monitored  at  a  ground  station.  Radio  communica¬ 
tions  are  provided  between  the  ground  station 
supervising  the  exercise  and  the  participating 
aircraft.  Potentially  hazardous  flight  condi¬ 
tions  are  automatically  detected  and  brought  to 
the  attention  of  a  training  supervisor  for 
appropriate  action. 

It  was  clear  that  if  a  common  performance 
measurement  system  could  be  developed,  the 
potential  for  a  classical  transfer-of-training 
(TOT)  design  and  associated  statistical  analyses 
for  the  TEE  would  be  enhanced.  Thus,  for  the 
first  time  it  appeared  feasible  to  measure 
transfer  of  performance  frem  simulator  to  ACM 
flight  conditions  as  represented  on  the  TACTS. 

The  ACMS  is  a  one-of-a-kind  device  and  joint 
operation  with  TACTS  yields  quantitative  air 
combat  maneuvering  performance  data  which  are  a 
direct  function  of  pilot  training  and  performance. 


The  first  step  in  the  TEZ  was  to  examine  the 
TACTS  range.  TACTS  was  developed  to  provide  air¬ 
crews  with  an  objective  means  for  improving 
missile  envelope  recognition.  The  TACTS  provides 
training  in  a  realistic  but  controlled  environ¬ 
ment.  The  trainee  receives  limited  real-time 
airborne  feedback  and,  later,  more  thorough 
"debrief"  feedback  concerning  the  effectiveness 
of  weapons  firing  which  a.e  simulated  by  the 
TACTS  computers  at  the  ground  station. 

The  TEE  program  also  was  timely  in  that  it 
was  able  to  directly  support  Commander  Fighter 
Wing  One  (COMFITWING  ONE)  personnel  who  were  in 
the  process  of  developing  a  PMC  for  the  TACTS 
range.  The  measurement  system  being  developed 
for  TACTS  was  entitled  The  Readiness  Index  Factor 
(RIF).  It  was  based  upon  the  Readiness  Estima¬ 
tion  System  (RES)  first  developed  by  the  Center 
for  Naval  Analyses.  The  RIF  provided  a  measure 
of  the  spatial  relationship  of  two  or  more 
interacting  aircraft  and  considered  the  type  of 
weapons  employed. 

DATA  COLLECTION  AND  ANALYSIS 

TACTS  data  were  collected  and  analyzed  using 
the  RIF  during  two  Fleet  Fighter  ACM  Readiness 
Programs  (FFARPs),  which  comprise  a  syllabus  of 
’refresher"  exercises  (1.1,  2vl ,  Cv2,  etc.)  for 
fleet  squadrons  that  span  roughly  three  weeks. 

One  FFARP  involved  airborne  combat  exercises 
employing  friendly  F-4  aircraft  engage:'  with  F-5 
and  A-4  adversary  types.  The  other  FFARP  in¬ 
volved  F- 14  against  F-5  and  A-4  adversaries.  The 
data  indicated  that  the  RIF  was  sensitive  not 
only  to  differences  in  individual  aircrews,  but 
also  to  aircraft  variables  as  well. 

Figure  1  contains  one  comparison  of  such 
differences. 
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A  RIF  score  above  zero  indicates  that  an  air¬ 
craft  is  relatively  more  offensive  than  defensive 
in  relationship  to  an  adversary  aircraft.  A  nega¬ 
tive  RIF  score  indicates  the  friendly  aircraft  is 
in  a  defensive  posture.  The  RIF  scores  in  Figure 
1  indicate  that,  of  the  ten  F-4  aircrews,  only  one 
was  able  to  achieve  an  overall  positive  offensive 
average.  On  the  other  hand,  all  but  four  of  the' 
thirteen  F-14  aircrews  attained  an  overall 
Dositive  offensive  average. 

Aircrews  also  were  ranked  according  to  ob¬ 
jective  RIF  scores  and  these  scores  were  compared 
to  rankings  obtained  from  adversary  squadron 
pilots,  ACM  instructor  grade  sheets,  and  data  from 
the  "Blue  Baron"  data  collection  effort  conducted 
on  the  TACTS  range.  The  latter  effort  consisted 
of  a  combination  of  objective  and  subjective  data, 
including  parameters  such  as  radar  range,  system 
lock-on  parameters,  visual  sighting  and  kill /miss 
assessments.  In-depth  analysis  of  the  type  of 
shots  taken  revealed  that  ?  high  percentage  of 
missile  shots  fired  were  in  the  forward  hemisphere. 
Further  data  analysis  revealed  that  the  RIF  was 
net  sensitive  to  forward  hemisphere  f i . e . ,  all¬ 
aspect)  missile  firings. 

ALL-ASPECT  MANEUVERING  INDEX  (AAMI) 

As  a  result  of  the  first  data  collection,  it 
wrs  concluded  that  an  all -aspect  capability  had 
to  be  incorporated  in  the  RIF.  Using  the 
successful  elements  of  the  RIF,  an  All-Aspect 
Maneuvering  Index  (AAMI)  measurement  system  was 
designed  and  developed  for  the  next  data  col  lec¬ 
tion  series.  This  series  involved  data  collected 
from  craining  being  conducted  on  the  ACMS, 
involving  participation  of  a  number  of  F-14  and 
F-4  squadrons  operating  out  of  NAS  Oceana. 


The  AAMI  is  a  measurement  system  based  upon 
a  formula  that  incorporates  range,  antenna- 
train-ar.gle,  and  ang'ie-off-the-tail  as  the  major 
variables  which  define  the  spatial  relationship 
among  interacting  aircraft.  The  formula  is 
weighted  by  a  weapon  range  modifier  which 
accounts  for  the  influence  of  individual  weapon 
systems  such  as  the  AIM-9L,  AIM-7F,  and  AIM-9G. 

As  with  its  RIF  Predecessor ,  an  AAMI  score 
above  zero  indicates  that  an  aircraft  is  more 
offensive  tnan  defensive  in  relationship  to  an 
adversary  aircraft.  A  high  positive  score  is 
awarded  when  the  aircraft  achieves  an  optimal 
position  with  respect  to  the  weapons  load  it  has 
on  board.  In  addition  to  providing  graphical 
feedback,  the  AAMI  describes  a  range  of  numeric 
indicutors  such  as  time  to  fire,  time  to  first 
kill,  missile  type  information,  number  of  gun 
rounds,  etc. 

Figure  2  contains  AAMI  summary  data  from 
the  simulator.  All  simulator  engagements  cohered 
in  this  paper  pitted  an  aircrew  experienced  in 
ACM  against  a  computer-driven  adversary.  The 
data  in  Figure  2  reflect,  differences  due  to  air¬ 
craft  variables  quite  similar  to  the  differ- 
pnrps  shown  bv  previous  RIF  data  collected  on 
the  TACTS  range.  For  example,  F-14  aircraft 
consistently  outperform  F-4  aircraft  against  ad¬ 
versary  aircraft  in  a  simulated  environment.  The 
individual  differences  and  performance  variations 
among  the  three  F-4  aircrews  and  the  two  F-14 
aircrews  have  been  assessed  in  light  of  the 
historical  records  supplied  by  each  aircrew.  One 
point  stands  clear  from  an  examination  of  the 
data  in  Figure  2:  Post-test  scores  are  much 
higher  than  Pre-test  scores.  This  indicates  that 
the  training  realized  from  40  ACM  engagements  in 
the  simulator  does,  in  fact,  improve  performance. 
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Another  analysis  of  F-14  simulator  data  from 
the  friendly  aircraft  point  of  view  ib  contained 
in  Figure  3.  The  Pre-test  data  point  is  an  aver¬ 
age  from  seven  aircrews  over  the  three  engagements 
comprising  the  Pre-test.  The  run  conditions  and 
number  of  engagements  experienced  were  identical 
for  both  the  Pre-test  and  Post-test.  All  simula¬ 
tor  runs  were  stopped  when  they  exceeded  three 
minutes  in  duration  or  when  2  successful  missile 
kills  were  achieved  by  either  the  friendly  or 
the  bogey.  Of  significance.  Figure  3  demonstrates 
a  substantial  increase  in  performance  from  Pre- 
to  Post-test.  The  syllabus  numbers  in  between 
the  Pre-  and  Post-test  represent  incremental 
variations  in  the  training  steps  applied  to  the 
participating  aircrews.  Each  data  point  in  the 
graph  represents  the  average  performance  over 
ten  engagements  for  each  syllabus:  step.  The 
number  of  aircrews  participating,  as  well  as  the 
total  number  of  engagements  in  each  syllabus 
step,  varied  across  the  syllabus  due  to  uncon¬ 
trollable  factors  such  as  aircrew  availability 
and/or  simulator  equipment  problems.  The  forty 
engagements  experienced  between  the  Pre-  and 
Post-test  varied  in  the  level  cf  di faculty  pre¬ 
sented  to  the  friendly  aircraft.  As  shown  in 
the  "run  conditions"  at  the  bottom  of  Figure  J, 
the  first  ten  engagements  in  the  syllabus 
(Step  030)  haci  the  friendly  aircraft  loaded  with 
a  rear  hemisphere  missile  against  a  "medium" 
level  of  difficulty.  A  dotted  line  indicates 
that  no  change  occured  from  the  previous  condi¬ 
tion  in  that  syllabus  block.  Note  that  a  de¬ 
crease  in  offensive  advantage  was  registered  be¬ 
tween  the  Pre-test  and  Step  030,  during  which  the 
friendly  fighter  lost  his  9L  all-aspect  missile. 

In  addition,  during  the  next  ten  engagements 
(Step  040)  the  adversary  difficulty  level  was 
increased  to  "high",  which  resulted  in  a  drop 
in  the  aircrew  AAMI  average  to  below  zero. 

That  is,  moving  from  a  "medium"  to  a  "high" 
difficulty  level  opponent  resulted  in  average 


aircrew  performance  droppiny  from  an  offensive 
to  a  defensive  posture. 

Further  analysis  of  F-14  simulator  data 
contained  in  Figure  3  demonstrates  a  substantial 
increase  in  performance  from  Pre-test  to  Post¬ 
test.  Switching  from  "high"  difficulty  level  to 
"medium"  difficulty  level  when  the  friendly  and 
adversary  aircraft  were  loaded  with  similar  wea¬ 
pons  on  the  same  type  of  aircraft  (both  F-14s) 
resulted  in  an  average  offensive  advantage  for 
the  inenaiy  aircrew  as  shown  by  the  data  puinl 
in  Step  050.  Increasing  the  level  of  difficulty 
from  "medium"  to  "high"  while  maintaining  the 
same  aircraft  and  weapon  relationships  resulted 
in  a  decrease  in  average  Herformana  below  the 
offensive  (zero)  score  line,  once  again  as  shown 
by  the  data  point  for  Step  060. 

The  F-4  data  depicted  in  Figure  4  again 
indicate  that  the  AAMI  reflected  relatively  high 
offensive  performance  in  response  to  an  adver¬ 
sary  at  a  "medium"  difficulty  level  when  con¬ 
trasted  to  performance  against  a  "high"  difficulty 
level.  For  example,  a  lower  adversary  difficulty 
level  at  syllabus  Step  010,  in  comparison  to 
+  n associated  with  an  increase  i* 
the  average  offensive  score.  When  the  hiyh  ad¬ 
versary  difficulty  level  is  introduced  at  sylla¬ 
bus  Step  031,  a  performance  decrement  is  noted. 

The  data  from  the  last  twn  syllabus  steps  and  the 
Post-test  data  suggest  that  a  learning  asymptote 
may  have  occurred.  Six  aircrews  participated 
in  the  Pre-test:  although  again  the  number  of 
aircrews  as  well  as  the  number  of  engagements 
in  each  syllabus  step  fluctuated  due  to  uncon¬ 
trollable  factors.  As  with  the  F-14  data  in 
Figure  3,  the  Post-test  scores,  considered  in 
relationship  to  the  average  scores  and  corres¬ 
ponding  standard  deviations  achieved  in  other 
syllabus  steps  and  the  Pre-test,  suggest  that 
learning  has  taken  place. 
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An  examination  of  a  single  F-4  aircrew  is 
detailed  in  Figure  5  as  an  example  of  ‘he  type  of 
data  available  from  the  ACM  Per formant  a  Measure¬ 
ment  System  (PMS).  The  average  score  is  a  com¬ 
posite  measure  which  can  be  easily  broken  into 
other  types  and  combinations  of  metric  indicators. 
In  this  example,  the  Pre-test  average  score  is 
-22.4,  while  the  Post-test  performance  average  in¬ 
creases  to  *12.29.  At  the  same  time,  the  average 
engagement  length  decreases  markedly  from  184 
seconds  to  8?  seconds  from  Pre-Test  to  Post-test. 
The  Pre-test  also  demonstrates  another  interesting 
fact.  In  the  beginning  of  the  session,  the 
friendly  fighter  lost  once  but  never  won  a  Pre¬ 
test  engagement.  At  the  end  of  the  simulator 
training,  the  fighter  won  all  three  Post-test 
engagements.  The  initial  simulator  learning 
period  (Step  010.  the  f i ^ c  ten  engagements  after 
the  Pre-test)  also  ref1  -cts  vhe  fact  that  the  air¬ 
crew  was  learning  the  syllabus  objectives.  As 
demonstrated  in  Figure  5,  the  last  five  engagements 
of  each  syllabus  Step  (010(2)  and  040(2))  resulted 
in  the  fighter  increasing  his  average  offensive 
score,  decreasing  the  average  engagement  length 
and  winning  four  out  af  five  engagements  (on?  en¬ 
gagement  was  dropped  because  of  computer  problems). 
The  sa«*  learning  trend  is  illustrated  by  the 
data  from  the  concluding  simulator  learning  perio*4 
also  shown  in  Figure  S. 


DISCUSSION 

The  data  presented  in  this  paper  are  pre¬ 
liminary  in  nature.  The  point  of  this  paper  is  t.c 
bring  out.  the  fact  that  a  common,  objective  mea¬ 
surement  tool  has  been  developed  for  ACM,  and  that 
the  assumptions  underlying  the  system  have  been 
validated  via  actual  aircrew  training  data. 

The  ACM  performance  measurement  tool  can 
objectively  determine  differences  in: 

•  Aircraft  Performance  (Friendly  and  Bogey) 

•  Weapons  System  (AIM-91,  -^G,  -7F,  Guns) 

•  Aircrew  (Type  and  Background) 

•  Tactics 

•  Training  ( r  gu i pr'tnt .  Syllabi) 

Examination  of  the  data  has  been  extensive, 
ranging  from  initial  scattcrplots  of  relevant 
individual  performance  points,  to  variance  and 
cot » elat  ima  1  analyses  to  determine  the  signifi¬ 
cance  of  grouped  data  points  jnd  i rs?et\u  t  ions . 

This  examination  process,  including  review  by 
Subject  matter  exerts,  has  verified  the  validity 
and  reliability  of  the  system  as  well  as  the 
fsinlaew-nta  1  a-.Sunp*  i ons  of  the  PMS. 

A  difficul*  question  is  'Does  the  ^;ta  rol- 
Inted  and  analysed  t»us  far  indicate  that  the  air- 
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SIMULATOR  DATA  F-4  vs.  THREAT  1  (example- AIRCREW  #18) 


crews  are  learning  to  fly  the  simulator?”.  The 
answer  is  that  they  are,  indeed,  learning  some¬ 
thing.  The  scores  improve  and  the  variance  de¬ 
creases  in  accordance  with  pertinent  syllabus 
steps  and  in  response  to  modification  in  training 
•conditions.  Clearly  there  are  "tricks"  that  can 
be  learned  to  permit  one  experienced  simulator 
flyer  to  beat  another  simulator  aircrew.  And, 
given  enough  time,  some  of  these  tricks  may  possi¬ 
bly  be  adapted.  For  example,  it  is  possible  for 
an  aircrew  to  enhance  conditions  under  which  the 
computer-dri ven  bogey  becomes  increasingly  vul¬ 
nerable,  such  as  by  flying  at  very  high  angles 
of  attack  or  below  200  feet  in  altitude.  How¬ 
ever,  PSI  personnel  with  ACM  experience  have  been 
operating  and  "flying"  the  simulator  for  over  two 
years  during  and  before  this  study.  In  our 
opinion,  forty  simulator  engagements,  conducted 
under  varying  syllabus  conditions,  does  not  per¬ 
mit  an  aircrew  enough  time  to  "learn"  to  trick 
the  simulator.  Rather,  the  data  collected  thus 
far  and  the  feedback  from  the  aircrews  them¬ 
selves  strongly  suggest  that  more  substantive 
learning  is  taking  place. 


A  more  difficult  question  is  "Do  the 
patterns  learned  in  the  ACM  simulator  transfer  to 
the  TACTS  range?".  The  answer  to  this  question 
is  being  vigorously  pursued.  All  data  collection 
efforts  are  being  conducted  on  a  nrt-to- interfere 
basis.  It  is  difficult  to  carry  out  an  experi¬ 
mental  paradigm  on  a  simulator  that  is  being  used 
for  operations!  training.  It  is  deeidely  more 
difficult  to  apply  e*perir-nta!  rigor  in  the  TAC’S 
range  environment .  f*perimental  design  techniques 
and  statistical  controls  are  being  employed  to 
compensate  where  possible. 


Data  are  presently  being  collected  on  the 
TACTS  range  as  well  as  from  the  ACMS  to  answer 
both  questions.  These  questions  have  strong 
implications  for  R&D  investigations,  for  trailing 
system  design  efforts,  and  for  combat  readiness 
issues . 
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ABSTRACT 

The  Visual  Technology  Research  Simulator  (VTRS)  at  the  Naval  Training  Equipment  Center 
was  used  to  study  the  effects  of  six  factors  on  carrier-landing  training.  An  in-s:mulator 
transfer  design  was  chosen,  in  which  students  were  trained  under  various  conditions,  and 
then  tested  under  a  standard  condition  that  represented  maximum  realism.  The  experimental 
design  permitted  a  relatively  large  number  of  variables  to  be  studied,  using  a  lelat.ively 
small  number  of  student  subjects.  The  subjects  were  pilots  who  had  no  prior  carrier-landing 
experience:  16  resent  graduates  of  Air  Force  T-33  training,  and  16  highly  experienced  Navy 
P-3  pilots.  Factors  investigated  were  field-of-view.  scene  detail,  platform  motion,  descent- 
rate  cuing  and  training  task  fstraight-in  =>pproaches  vs.  circling  approaches).  Turbulence 
was  included  as  a  factor  and  pilot  type  (Navy  P-3  vs.  Air  Force  T-38)  was  also  included  as 
a  factor  to  control  this  source  of  subject  variability,.  After  trailing  under  a  certain 
factor-level  combination,  students  were  testeCLon  the  day,  wide  field-of-view,  circling  task 
with  motion  and  without  descent-rate  cuinq-V'ftesul ts  showed  that  the  simulator  and  training 
factors  generally  produced  either  small  differences  o."  no  differences  at  all  in  transfer 
effectiveness.,.  There  we^e  some  advantages  of  the  wide  field-of-view  and  high-detail  condi¬ 
tions,  but  these  effects  were  small  and/or  short-lived,  generally  disappearing  after  a  few 
transfer  trials.  Training  with  straight-in  approaches  resulted  in  transfer  performance 
that  was  equal  to  or  better  than  that  produced  by  training  with  circling  approaches.  There 
were  no  motion  or  descent-rate  cuing  effects  on  the  transfer  tasK 


INTPODUCTION 

The  Visual  Technology  Research  Simulator 
(VTRS)  at  the  Naval  Training  Equipment  Center 
(NTEC),  Orlando,  Florida  is  desianed  for  research 
on  flight  simulator  requirements  for  training  and 
skill  maintenance.  The  VTRS  consists  of  a  fully 
instrumented  Navy  T-2C  jet  trainer  cockpit,  a  six- 
degree-of- freedom  synergistic  motion  platform  and 
a  wide  angle  visual  system  that  can  project 
computer- generated  images  onto  a  spherical  screen. 
The  visual  system  is  capable  of  displaying  images 
via  target  and  background  projectors  subtending 
50  degrees  above  and  30  degrees  below  the  pilot's 
eye  level  and  can  display  160  degrees  of  horizontal 
field.  *' 


from  the  phase  two  experiment  reported  here  is 
relevant  to  the  design  of  simulators  for  under¬ 
graduate  pilot  training.  The  experiment  involved 
pilots  with  no  carrier-landing  experience  who 
trained  in  the  simulator  under  various  conditions 
representing  leveTs  of  several  simulator  equip¬ 
ment  and  training  factors.  The  pilot,  were  then 
tested  in  the  simulator  in  it<  high  f  delity 
configuration.  An  in-simulator  transfer-of- 
traininq  paradigit  was  used  to  study  the  effects 
of  sir.  factors  plus  a  pi  lot- type  factor  on 
carrier-landing  training.  A  total  of  32  pilots 
with  no  prior  carrier-landing  experience  were 
involved  in  the  experiment.  Sixteen  were  recent 
graduates  of  Air  Force  T-38  training  and  16  were 
experienced  Navy  P-3  pilots. 


A  major  effort  at  7TRS  has  involved  research 
to  define  simulate**  requi rements  for  the  carrier- 
landing  task.  There  is  a  need  to  investigate  a 
large  number  of  visual  and  oth°r  simulator  fea¬ 
tures  having  substantial  cost  implications.  A 
research  program  was  planned  around  the  holistic 
experimental  philosophy  and  paradigm  proposed  by 
Simon  ::*-  1  wh i ch  stresses  the  important  e  of 
studying  as  many  factors  of  interest  as  possible 
within  a  single  experiment.  The  first  phase  of 
this  program  consisted  of  performance  studies  in 
which  the  effect  or  various  simulator  components 
on  experienced  pilots  in  the  simulator  was  exam¬ 
ined.  !f  ,  :  '  The  second  phase  consisted  of  a 
quasi-transfer  study  in  which  the  simulator  was 
both  the  training  and  the  criterion  device.  Rhase 
three  will  eventually  *nclude  a  Simula  tor- to- f ield 
transfer  study  involving  actual  flight  tests. 

This  report  presents  results  from  the  phase  two 
i  o  *  s  i  mu  1  a  to r  t  ra  ns  f  e  r- o  f  - 1  ra  i  n  i  ng  e  *  pe  r  i  men  t . 

The  information  obtained  from  P*vre  one 
expert ments  was  directly  relevant  to  the  desig? 
of  simulators  for  experienced  pilot  skill  main¬ 
tenance  and  tranr.it ion  training.  The  inf.srm.at ion 


METHOD 


Task 


The  mission  was  a  daytime  carrier  approach 
and  landing  on  the  deck  of  the  aircraft  carrier 
Forrestal.  The  carrier  was  moving  at  20  knots 
with  a  zero  effective  crosswind  over  the  landing 
deck  and  knots  relative  wind  down  the  deck. 

The  mission  for  this  experiment  was  restricted  to 
include  only  the  final  turn  of  the  full  cir  ling 
maneuver  as  well  as  the  final  approach  and  landing. 

Factors  and  Levels 

A  large  r.unher  of  factors  potentially  affect¬ 
ing  the  carrier- landing  cues  were  tentatively 
selected  as  candidates  for  study.  These  were 
pared  down  bv  a  panel  of  engineers  and  psycholo¬ 
gists  ini.  a  set  of  factors  which  wore  invest  1  - 
gated  earlier  in  performance  experiments. 

Tec.  is  tops  regarding  factors  to  W  included  in  this 
in- simulator  t**ans fer-of-tra irinq  experiment  were 
*  axed  ;  ! r 1 1  v  on  results  from  the  «  ■erfo*— wince 
studies.  Other  cons ide*at ions  were  the  potential 
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cost  impact  of  factors  and  the  potential  training 
effects  and  interactions  involving  factors  falling 
in  the  category  of  training  aid  or  type. 

"High"  and  "low"  factor  level  settings  were 
chosen  in  order  to  bracket  the  reasonable  range  of 
interest.  For  the  equipment  factors,  the  high 
levels  were  general  V  set  at  the  highest  attainable 
while  the  low  levels  were  chosen  to  be  the  most 
degraded  form  of  the  factor  likely  to  be  employed 
operational ly .  Other  factor  levels  also  bracket 
the  range  of  interest,  but  do  not  nece^sarf ly  con¬ 
form  to  "high"  and  "low"  definitions.  A  summary 
of  the  factors  involved  in  this  experiment  is 
given  in  Table  1. 

Field-of-View.  The  field-of-view  high  level 
was  a  160  "degree  horizontal  by  80  degree  vertical 
wide-angle  display/14'  This  wide  field-of-view 
is  costly  and  is  representative  of  that  currently 
available  for  carrier-landing  training  only  on 
multi-task  trainers  such  as  tne  2B3S  and  the  F- 14 
Wide-Angle  Visual  System  (WAVS).  The  field-of- 
view  low  level  was  a  plus  or  minus  ?4  degrees 
horizontal  by  -27  degrees  to  9  degrees  vertical 
narrow-angle  display.  This  narrow  field-of-view 
is  representative  of  the  lower  cost  Night  Carrier 
Landing  Trainers  (NCLTs)  used  for  F-4,  F - 14 ,  A-6, 
A- 7,  and  S-3  transition  training. 

Scejne  Deta^i  1 .  The  ship  detail  for  the  high 
detail  scene  was  a  daytime  solid  model  computer- 
generated  image  (CIO)  carrier  whose  surfaces  were 
defined  by  985  edges;4 /he  daytime  scene  included 
a  background  seascape  colored  a  uniform  blue  and 
a  well  defined  horizon.  Brightness  levels  were 
approximate ly  2.80,  0.50,  and  0.16  foot  Lambert 
(fL)  for  the  ship,  spa  and  sky,  respectively.  The 
detail  level  was  representative  of  that  available 
from  typical  daytime  CIG  systems  costing  several 
million  dollars.  The  low  level  of  scene  detail 
was  an  image  of  a  night  point-light  CIO  carrier 
consisting  of  137  lights.  It  contained  all  deck 
outline,  runway,  centerline  and  drop  lights.  The 
background  was  dark  with  no  visible  horizon.  This 
display  is  representative  of  a  night  CIG  system 
costinq  less  than  a  million  dollars  and  used  on 
several  Navy  NCLTs. 

I'otion.  A  si x-deqree-of-freedom  48-inch 
syr.erqistic  motion  platform  was  fully  operational 
for  the  high  level,  and  was  stationary  for  the  low- 
level  of  this  factor.  This  platform  is  similar 
to  those  on  the  Navy's  27  T-2C  Instrument 
Trainers  (device  2F101)  used  in  Undergrade  Pilot 
Training  (UP;)  except  that  VTRS  computation  rates 
are  higher  for  reduced  cuing  time  lag.  While  it 
is  representati ve  of  many  cider  platforms  on 
existing  trainers,  it  does  not  have  the  lew  noise 
and  improved  response  of  new  platforms.  An 
attempt  was  made  to  fine  tune  the  operational 
pi  a  t  forte  for  optimal  responsiveness  for  the 
rier  landing  tas*  by  setting  gains  at  7.5,  2.8, 
an<5!  1.2  for  lateral,  heave,  and  pitch  cuing, 
respectively.  R#U,  thrust,  and  yaw  cuing  ’ains 
remained  at  1.0. 

Approach  _Tjpe .  Pilots  performed  their 
training  trials  with  either  straight- in  appr^cht^ 
or  circling  approaches,  lor  the  circling  approach 
the  Aircraft  was  initialised  abeam  the  LS0  plat- 
fere  at  5700  fr$n  the  \h  g  and  at  600  feet 
of  altitude  in  the  approach  configuration  (full 


flaps,  speed  brake  out,  hook  and  wheels  down  and 
15  units  angle-Gf-attack) .  Fuel  was  fixed  at 
3200  pounds  to  give  a  gross  aircraft  weight  of 
10,000  pounds.  A  trial  consisted  of  the  final 
turn,  final  approach  and  attempted  landing. 

The  straight-in  approach  was  initialized  with 
the  aircraft  11,990  feet  be,  ind  the  ship  and  4150 
feet  to  the  left  of  the  runway  centerline.  The 
initial  altitude  was  400  feet  with  the  aircraft 
in  the  approach  configuration.  The  aircraft  was 
trimmed  for  straight-and-level  flight  in  boLh 
initial  approach  conditions.  The  straiqht-^n 
approach  was  defined  specifically  to  provide  task 
requirements  similar  to  the  circling  approach  hut 
with  the  ship  in  view  at  all  times  under  the  nar¬ 
row  field-of-view  condition.  Thus  the  aircraft 
was  set  to  the  left  of  the  runway  centerline  and 
headed  at  18  degrees  to  the  right  of  the  ship 
centerline  heading.  (At  25  degrees  of  aircraft 
heading  relative  to  ship  centerline  heading  the 
ship  would  go  out  of  the  narrow  field-of-view.) 
Pilots  were  instructed  to  fly  tlvs  modified 
straight-in  approach  straight-and-level  out  of 
the  initial  condition  until  approaching  the  run¬ 
way  centerline,  then  execute  a  turn  end  rollout 
on  the  runway  centerline. 

This  factor  was  included  in  the  experiment 
because  of  the  hypothesis  that  the  daytime  mission 
can  be  trained  adequately  with  (modified)  straight- 
in  approaches.  This  hypothesis  was  supported  by 
results  presented  in  r ^  and  by  preliminary  obser¬ 
vation.  If  this  hypothesis  were  true,  it  could 
have  implications  for  the  field-of-view  question 
since  a  wide  field-of-view  is  not  required  to 
keep  the  ship  visible  to  the  pilot  during  straight- 
in  approaches. 

FLOLS  Rate  Cuing.  The  conventional  version 
of  the  TLOLS  display  defined  one  level  of  this 
factor.  The  other  level  involved  the  uie  of 
vertical  bars  displayed  w- th  the  FLOLS  which  pre¬ 
sented  glideslope  rate  of  change  information  co 
the  pilots.  This  information  was  presented  to  the 
pilots  in  ’command"  fashion,  that  is,  the  bar 
height  could  be  interpreted  directly  in  terms  of 
action  required  to  change  to  the  desired  vertical 
velocity.  The  heiqht  of  the  bars  indicated  devi¬ 
ation  from  correct  vertical  velocity  relative  to 
current  glideslope  displacement  and  nulled  bars 
indicated  correct  vertical  velocity  for  any  glide- 
slope  position.  The  FLOLS  rate  cuing  factor  was 
included  in  the  experiment  on  the  basis  of  VTRS 
work,  which  indicated  a  large  improvement  in 
glideslope  control  with  rate  cuing  displays  for- 
experienced  carrier  pilots.  " *  * 1  The  latter  ref¬ 
erence  (Experiment  II)  include^  a  description  of 
the  algorithms  used  to  define  the  command  rate  bar- 
operation  . 

Turbulence.  Turbulence  was  included  in  the 
experiment  at  two  levels  to  allow  examination  of 
otner  factor  effects  under  two  difficulty  levels. 
The  two  levels,  no  turbulence  and  toe  highest 
amount  of  turbulence  under  which  operations  would 
continue  at  sea,  represent  the  ranee  of  expected 
real-world  turbulence.  Thn  main  effect  of  turbu¬ 
lence  was  also  of  some  interest  in  this  experiment 
as  a  training  variable.  Turbulence  was  generated 
in  the  form  of  "winds"  acting  on  the  longitudinal, 
lateral,  and  vertical  aircraft  axis.  These  winds 
were  pseudo- random  sire  waves  which  were  generated 
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TABLE  1.  SUMMARY  OF  EXPERIMENTAL  FACTORS  AND  LEVFLS 


LEVEL  SETTINGS 


FACTORS 

"low" 

"hi gh" 

Field-of-view 

-27  degrees  to 

9  degrees  vertical, 

*24  degrees  horizontal 

*  -30  degrees  to 

50  degrees  vertical , 
*80  degrees  horizontal 

Scene  Detail 

Night:  point-light 
ship 

*  Day :  sol i d  surface 
ship 

Motion 

Fixed  base 

*  Si x-degree-of  freedom 

Approach  Type 

♦Circling  approach 

Modi fied  straight-in 
approach 

FL0LS  Rate  Cuing 

♦Conventional  FL0LS 
display 

FL0LS  display  with 
"command"  rate  cuing 

Turbulence1 

Close  to  maximum 
flyable 

No  turbulence 

p'lot  Type 

Air  Force  T-38 

Na\ /  P-3 

*  I nd i cates  setting  for  the  transfer  test  conf iqurati on 


’Turbulence  was  set  at  half 
the  transfer  test. 


by  the  summation  of  pure  sine  waves  of  various 
frequencies  and  amplitudes.  The  RMS  values  of  the 
winds  used  were  3.00,  1.25,  and  3.00  ft/sec  for 
the  longitudinal,  lateral,  and  vertical  dimensions, 
respectively.  These  values  produced  e  fairly  large 
amount  of  turbulence  judged  to  be  near  the  limits 
for  saf*1  operation.  Frequency  and  amplitude  val¬ 
ues  used  are  (liven  in 

Pilot  Type.  Pilots  for  this  experiment  were 
selected  from  two  populations  without  carrier- 
landing  experience.  One  group  was  made  up  of 
recent  graduates  of  Air  Force  T - 33  training  and 
the  other  group  was  comprised  of  experienced  Navy 
P-3  pilots.  The  pilot  group  factor  was  included 
explicitly  ir  the  experiment  so  that  the  expected 
large  source  of  subject  variance  resulting  from 
differences  between  the  groups  could  be  directly 
estimated  and  partial  led  out  of  the  results. 

Factors  Held  Constant.  A  number  of  factors 
investigated  earlier  were  held  constant  in  this 
training  experiment.  l'*:’  C I G  images  of  both 
ship  and  Fl.OLS  were  used  throughout  the  experi¬ 
ment.  The  TV  line  rate  was  1025  and  engine  com¬ 
putations  were  done  at  30  Hz.  Visual  lag  w«s  at 
the  system’s  optimum  100  msec  and  the  0-seat  was 
off. 

Transfer  Configuration 

After  completing  their  training  sessions  a 1 1 
pilots  transferred  to  un  in-sirnjlator  maximum 
realism  version  of  the  daytime  circling  carrier- 
landing  task.  The  initial  position  for  the 
circling  approach  was  the  sane  as  that  used  for 
circling  approaches  during  training.  A  wide 
field-ot-view  was  .resent  along  with  the  daytn>e 
scene,  motion  on,  convent iona’  FL0LS  and  an 


the  "low"  level  setting  for 


intermediate  level  of  turbulence.  RMS  values  for 
the  "winds"  used  during  training  were  halved  for 
the  transfer  test.  This  amount  of  turbulence  was 
judged  to  be  somewhere  between  small  and  moderate 
in  size. 

Pilots 

Pilots  for  this  experiment  were  volunteers 
from  two  populations  with  no  carrier-landing 
experience.  One  group  of  16  pilots  was  made  up 
of  recent  graduates  of  Air  Force  T-38  training. 

All  the  pilots  in  this  group  had  approximately 
200  hours  of  total  flight  time  with  about  100 
hours  of  flight  time  in  the  T-38  in  the  six  months 
prior  to  the  experiment.  The  other  group  was  com¬ 
prised  of  16  experienced  Navy  P-3  pilots.  This 
group  averaged  1550  hours  of  total  flight  time  but 
varied  considerably  in  overall  experience  with  a 
standard  deviation  of  529  and  a  range  of  600  to 
2530  flight  hours.  Experience  in  the  six  months 
preceding  the  experiment  averaged  229  hours,  all 
in  the  P-3. 

Performance  Measures 

All  of  the  surimary  measures  described  and 
collected  in  r:‘  were  also  collected  for  the 
present  experiment.  Due  to  space  limitations, 
analyses  for  only  one  measure,  which  essentially 
summarizes  final  approach  glideslope  oerforrance, 
are  presented  here.  A  percent  tine-wi thi n- 
tolerance  measure  for  the  final  approach  was  used 
to  summarize?  final  approach  glideslope  perfor¬ 
mance.  This  score,  referred  to  as  the  glideslope 
tracking  score,  was  calculated  from  3000  feet  to 
the  ramp  c.nd  used  a  tolerance  band  of  *.3  degree. 
This  tolerance  was  recommended  by  Navy  Landing 
Signal  Officers  (LSOs)  and  represents 
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approximately  plus  or  minus  one  "meatball"  of 
deviation  of  the  ^LOLS  display.  Results  for  other 
scores  are  discussed,  including  a  wire  (longitud¬ 
inal)  accuracy  metric  for  assessing  touchdown 
quality.  This  score  is  similar  to  Brictson's 
Landing  Performance  Score  (LPS),  but  it  is  based 
on  absolute  deviation  from  the  desired  touchdown 
point  rather  than  wire  catch  per  se.  Touch¬ 
downs  resulting  in  a  three-wire  catch  were 
assigned  a  score  of  100,  while  touchdowns  more  than 
100  feet  behind  the  one-wire  or  beyond  the  four- 
wire  were  assigned  scores  of  zero.  Touchdowns 
inside  these  limits  were  assigned  scores  linearly 
between  zero  and  100. 

Procedures 

The  experiment  consisted  of  40  training 
trials  and  16  transfer  trials.  Before  flying  any 
trials,  pi  lots  were  given  approximately  one  and 
one-half  hours  of  instruction  in  carr ier- 1 andi nq 
procedures  in  the  form  of  a  briefing  and  an  in¬ 
structional  videotape.  They  then  flew  two  three- 
minute  fami  1  iarization  fights  in  the  simulator 
befure  commencing  with  the  experimental  training 
trials.  Instructional  feedback  was  given  after 
each  training  trial  by  a  VTRS  staff  member  and  an 
"automatic  LSO"  was  used  throughout  the  experiment 
to  give  calls  during  the  flights.  (A  V0TRAX  voice 
generation  system  was  driven  by  software  developed 
from  a  modified  version  of  the  model  described  in 
!!  .  Instructional  feedback  was  not  oiven  during 
the  transfer  test  trials. 

Exner  i  men  tc  i  Pes  i_gn 

A  transfer-of- tra i ni ru;  paradigm  was  super¬ 
imposed  on  the  basic  experimental  design  which  was 
an  adaptation  of  National  Bureau  of  Standards, 

Plan  4.7.16.  Each  pilot  performed  40  training 

trials  under  one  of  the  condition0  of  the  bafc 
design  followed  by  16  transfer  trials  in  the  simu¬ 
lator  on  the  high  fidelity  transfer  configuration. 
The  basic  design  was  one-fourth  of  a  fully 
crossed  2  design  resultinq  in  32  experimental 
conditions  and  31  estimable  effects. 

All  main  effects  are  confounded  only  with 
three-way  or  higher  interactions  as  are  15  of  the 
21  two-way  interactions.  The  other  six  two-way 
interactions  are  confounded  in  strings  of  two  each 
and  the  remaining  six  estimable  terms  in  the  basic 
design  reprecent  strinqs  of  three-way  interactions . 
As  the  oasic  design  was  repeated  across  trials, 
trial  effects  are  also  fully  estimable.  The  two- 
way  interactions  judged  least  likely  to  be  impor¬ 
tant  a  priori  were  assigned  to  the  strings  of  con¬ 
founded  two-factor  interactions.  Implicit  in  the 
use  of  this  fractional  factorial  design  is  the 
assumption  that  three-way  and  higher-order*  inter¬ 
actions  will  generally  be  negligible.  Each  of  the 
estimable  effects  in  the  basic  design  is  also  con¬ 
founded  with  the  subject  effects  defined  by  the 
groupings  involved  in  a  particular  comparison. 

RESULTS 

Table  2  presents  an  analysis  of  variance 
summary  of  transfer  test  trials  for  the  glideslope 
tracking  score.  This  analysis  estimates  effects 
averaged  across  all  16  transfer  trials  which  are 
represented  by  two  blocks  of  eight  trial  means. 

Vain  effects  were  tested  separately  «. >  were  the 


fieid-of-view  by  approach  type  and  scene  detail  by 
approach  type  interactions  which  were  considered 
important  a  priori .  All  other  estimable  inter¬ 
actions  were  examined  individually,  but  in  the 
absence  of  strong  evidence  for  an  effect,  were 
included  in  various  omnibus  terms.  The  basic 
"residual"  term  for  an  analysis  of  variance  was 
created  from  the  sum  of  the  two-  and  three-way 
string  terms.  The  sum  of  two-way  interactions  not 
involving  pilot  type  and  the  sum  of  two-wav  inter¬ 
actions  involving  pilot  type  wens  testec  against 
the  basic  residual  term.  If  tnese  effects  were 
not  significant,  they  were  combined  with  the 
residual  term  to  form  a  residual  estimate  against 
which  all  other  effects  were  tested.  Thus,  for 
example,  the  F  ratios  shown  for  the  single  degree- 
of-frecdom  effects  in  Table  2  have  a  22  deqree-of- 
freedom  denominator  whose  sums-of-sqjares  is  from 
all  the  indicated  sources. 

The  entire  transfer-of-trai ning  design  may 
be  thouqht  of  as  a  snecial  case  of  a  repeated 
measures'  design  with  observations  repeated  on  a 
trials'  factor.  1 " '  In  this  sense,  the  residual 
estimate  represents  an  estimate  of  subjects  within 
groups  error.  Trial  blocks  were  *ested 
against  a  term  comprised  of  all  estimate  three-way 
and  higher  interactions  involving  blocks  which  is 
then  an  estimate  of  error  within  subjects.  Two- 
factor  interactions  involving  blocks  were  test  d 
omnibus  fashion  against  this  same  term. 

Figure  1  graphically  describes  the  experi¬ 
mental  results  for  the  main  effects  ori  glideslope 
tracking  performance.  The  numbers  in  parentheses 
in  F-igure  1  above  a  block  of  trials  for  the 
effects  represent  the  results  of  an  analysis  of 
variance  for  tnat  block  of  trials.  The  numbers 
are  the  ranks  of  the  sizes  of  the  effects  (within 
the  31  estimable  effects)  of  the  basic  design. 

Thus,  for  example*  approach  type  had  the  largest 
effect  in  the  first  Mock  of  training  trials, 
field-of-view  ranked  fifth,  etc.  An  effect  can 
rank  high  by  chance  alone,  of  course,  and  such 
things  as  stability  over  time  and  effect  size 
must  also  be  taken  into  account  when  judging  the 
meaningfulness  and  reliability  of  an  effect.  On 
the  other  hand,  if  an  effect  size  does  not  rank 
very  high,  it  probably  is  neither  meaningful  nor 
real  (ince  it.  cannot  be  differentiated  fr on  noise. 

Table  2  and  Figure  1  indicate  a  large  dif¬ 
ference  between  pilot  types  on  the  transfer  task. 
This  was  not  unexpected  because  of  the  differing 
flight  backgrounds  of  the  groups  and  there  is 
little  interest  in  this  effect  per  se  other  tnan 
to  indicate  that  this  large  source  of  >ub.iect 
variance  was  successfully  controlled.  The  vide 
field-of-view  results  in  a  sustained  transfer 
a 'vantage  of  about  6.3  percent  more  time  within 
-.3  of  the  desired  glideslope.  This  effect  is 
the  fourth  largest  in  both  blocks  of  transfer 
trials  and  accounts  for  8.4  percent  of  the  experi¬ 
mental  variance  on  the  transfer  trials.  These 
results  are  in  the  same  direction  as  those  obtained 
■'n  although  significance  for  transfer  trials 

wac  not  obtained  in  that  experiment.  Although  the 
field-of-view  transfer  effect  appears  real  here, 
it  should  be  mentionel  that  there  was  no  field-of- 
view  transfr-  effect  on  touchdown  performance.  It 
is  felt  that  a  glideslope  effect  which  is  not 
reflected  in  touchdown  performance  should  be 
regarded  as  small.  Thus,  in  an  overall  sense,  the 
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TABLE  2.  ANALYSIS  OF  VARIANCE  FOR  GLIDESLOPE  TRACKING 
ON  '  N1SFER  TRIALS 


Source  of 

LEVELS 

Mean 

Variance 

Hijgh 

Low 

df 

Di fference1 

F 

Field-of-view 

Wide 

Nar 

1 

6.3 

(8.4)’ 

6.45* 

Scene  Detail 

Day 

Night 

1 

4.6 

(4.6) 

3.48 

Motion 

On 

Off 

1 

3.2 

(2.2) 

1.65 

Appioach  Type 

St.  In 

Ci  rc 

1 

2.5 

(1.4) 

1.05 

FLOl.S  Rate  Cue 

Cuinq 

No  Cue 

1 

-3.9 

(3.3) 

2.49 

Turbulence 

Calm 

Winds 

1 

2.8 

(1.6) 

1.26 

Pilot  Type 

Nav  P-3 

AF  T-3P 

1 

-7.6 

(12-3) 

9.37** 

F0V  X  App.  Type 

1 

(-) 

0.49 

S.  Dt.  1  X  App.  Type 

1 

(-) 

0.44 

2-Factor  Int  (No  Pi  1 )  ' 

7\ 

Pe- 

(9.7) 

1.14 

2-Factor  Int  (Pi  1  )4 

6) 

>  sid- 

(8.1) 

1.10 

2+3  Way  Strings 

9/ 

ual 

(11.0) 

Blocks  (8  Trials) 

1 

5.9 

(7.5) 

8.42** 

2-Factcr  Int  (Blocks) 

7 

(7.4) 

1.18 

3-Factor  Int  (Blocks) 

24 

(21.4) 

Grand  Mean 

34 . 8 

Std.  Err.  Difference 

2.3 

Std.  Deviation 

6.5 

‘Mean  of  observations  takei  under  high  level,  minus  mean  of  observations 
taken  under  low  level  of  actor. 

•Values  in  parentheses  ar:  percent  variance  accounted  for  in  the  experi¬ 
ment.  Percents  less  than  ].D  are  shown  by  a  dash  (-). 

'Two-factor  interactions  no  involving  pilot  type. 

‘‘Two-factor  interactions  i.nvolvinq  pilot  type. 

*p  •  .05 

**p  •  .01 


TURBULENCE  APPROACH  TYPE  SCENE  DETAIL  PILOT  TYPE 


Block*  3  kick*  Block*  Bloc** 


ELOLS  rate  cuing 


1  ?  3  <•  s  •  i:34S«  WITHIN  A  BLOCK  OF  TRIALS 


Block*  Block* 

SLOCKS  1  4  (Training  Trial*) 

Fiqure  1.  Main  Effects  for  G1  ideslope  ar£  io  trial  means 

Tracking  Score:  Percent  '.3  of  BLOCKS  5  6  (Tram)rr  Tnai»> 

Desired  for  3000  Feet  to  Ranp.  ARE  8  trial  MEANS 
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field-of-view  impact  on  transfer  performance  was 
not  large. 

Traininq  on  the  hiqh  level  of  scene  detail 
results  in  an  initial  advantaqe  on  the  transfer 
task.  As  Fiqure  1  indicates,  this  is  the  largest 
experimental  effect  on  the  first  block  of  transfer 
trials.  However,  by  the  second  block  of  transfer 
trials,  this  difference  has  essentially  dissipated 
and  the  overall  transfer  effect  is  not  significant, 
account inq  for  only  4.6  percent  of  transfer  task 
variance.  No  other  differential  transfer  effects 
were  evident,  although  the  approach  type  effect  is 
noteworthy.  Pilots  who  trained  with  modified 
straiqht-in  approaches  did  as  well  on  the  transfer 
task  as  chose  traininq  with  circlinq  approaches, 
'his  was  true  despite  the  fact  that  the  transfer 
task  included  a  circlinq  approach  and  those  pilots 
traininq  with  straiqht-in  approaches  had  not  flown 
the  circling  approach  prior  to  the  transfer  task. 
This  result  is  also  in  agreement  with  results 
given  in 

It  should  be  noted  also  that  the  FLOLS  com¬ 
mand  rate  cuinq  did  not  result  in  a  performance 
advantaqe,  even  during  training.  This  is  in  con¬ 
tradiction  to  results  given  in  ' "  which  show  a 
larqe  improvement  in  performance  with  the  rate 
cuinq  display  for  experienced  pilots.  The  pi  lots 
in  this  study  were  completely  novice  to  the  task 
and  apparently  d’d  not  have  the  skill  level  uerev 
sarv  to  effectively  utilize  the  rate  inf urmat inn 
at  this  stage,  at  least  not  with  the  command  dis¬ 
play  node  used  here.  Most  pilots  did  show  consid- 
eraole  general  improvement  in  performance  over  the 
length  of  the  experiment,  indicating  that  a  qreat 
deal  of  learning  was  taking  place  over  a  wide 
variety  of  conditions.  The  Air  Force  T-3R  group, 
in  particular,  had  readied  an  intermediate  pro¬ 
ficiency  level  by  the  end  of  the  experiment ,  an 
accomplishment  which  suggests  considerable  poten¬ 
tial  for  the  simulator  as  a  trainer  for  the 
carrier-landing  task. 

SUMMARY 

The  following  sumary  of  this  experiment 
includes  more  extensive  results  presented  in  : 
as  well  as  the  limited  results  presented  here. 

The  main  conclusion  to  be  drawn  fr^m  the  results 
is  that  equipment  li fferent ial  transfer  effects 
were  small  from  a  practical  point  of  view.  Trans¬ 
fer  landing  quality  was  not  affected  hv  u»w  factor 
other  than  pilot  type  and  approach  quality  was 
generally  only  temporarily  affected  by  equipment 
factors,  i.o.,  effects  had  essentially  disappeared 
after  eight  transfer  trials.  The  onlv  exception 
to  this  is  field-of-view  which  did  show  a  sus¬ 
tained  effect  on  glides  lope  tracking  performance. 
Although  final  approarh  effects  are  important  in 
their  own  right,  approach  effects  that  a**e  net 
reflected  in  touchdown  performance  rust  be 
regarded  as  small.  The  pilot  type  effect  was 
generally  larger  than  all  other  effects  combined 
and  the  only  other  factor  that  had  a  sizable,  sus¬ 
tained  transfer  effect  w^s  a  training  factor, 
approach  type. 

These  findings  have  important  implications 
for  simulator  design.  They  suggest  that  cost  an<, 
reliability  considerations  should  play  the  major 
role  in  selecting  the  dr  sign  for  an  undergraduate 
carrier- ’"anding  simulator.  Simulator  hardware 
advance  has  apparently  reached  a  stage--at  least 
for  the  carrier- landing  task --where  costly 


increases  in  fidelity  have  smaller  effects  than 
pilot  and  training  factors  and  are  producing  small 
gains  at  best.  The  findings  suggest  that  optimal 
cosi-effective  training  will  occur  with  a  rela¬ 
tively  low-fidelity  simulator  (as  defined  by  the 
low  levels  of  factor'  used  in  this  experiment)  and 
judicious  use  of  training  variables  and  schedules. 

I  ndi vidua 1  Factors 

A  brief  summary  of  results  follows  with  the 
factor  effects  listed  in  the  order  of  estimated 
overall  impact  on  the  transfer  task. 

Pilot  Type  had  by  far  the  largest  effect 
during  the*  transfer  trials  but  the  effect  was 
dependent  on  the  dimension  of  performance  measured. 
Air  Force  T-3S  pilots  did  better  than  Navy  P-3 
pilots  on  glideslope  t  acking  and  landing  wire 
accuracy,  but  P-3  pilots  did  much  better  on  final 
approach  lineup  tracking  and  angle-of-attack 
control . 

The  straight-in  approach  tyjpe  training 
resulted  in  better  final  approach  lineup  control 
on  the  transfer  task  despite  the  fact  that  the 
transfer  task  involved  a  circlinq  approach.  Pilots 
traininq  with  straight- in  approaches  had  lo  more 
time  witnin  the  lineup  tolerance  limit  on  the 
transfer  task  than  pilots  training  with  circling 
approaches.  Also,  the  circlinq  approach  training 
showed  no  transfer  advantage  for  other  final 
approach  scores  or  landing  accuracy. 

The  wide  field-of-view  resulted  in  some 
advantage  on  the  transfer  task  for  final  approach 
quality  but  not  landing  accuracy.  There  was  *  sus¬ 
tained  effect  on  glideslope  tracking  and  a  tempo¬ 
rary  effect  on  lineup  and  angle-of-attack 
tracking.  The  overall  effect  on  final  approach 
qualitv  after  eight  transfer  trials  was  small  at 
best . 

Uay..ine  scene  detail  training  conditions 
showed  no  transfer  advantage  over  night  scene  do  - 
tail  training  conditions  on  landing  accuracy. 

There  were  tiansient  effects  on  glideslope  and 
angle-of-attack  performance  with  the  daytime 
training  scenes  showing  the  advantage.  There 
appeared  to  be  a  lineup  performance  advantage  on 
transfer  for  the  daytime  training  scenes  but  onlv 
with  straight-in  training  approaches. 

The  presence  or  absence  of  motion  during 
traininq  did  not  make  a  difference  on  transfer. 

The  addition  of  rate  cuing  information  to  the 
FLOLS  did  not  result  in  a  transfer  advantage  tor- 
pared  tn  training  with  a  conventional  FLOLS. 

Con side rat  ions 


Two  things  should  be  kept  in  mind  when  con¬ 
sidering  these  results.  F.rst,  as  mentioned  ear 
lier,  subject  effects  defined  by  the  particular 
group  comparisons  were  confounded  with  experi¬ 
mental  effects.  Although  this  is  true  for  any 
transfer-of-traininq  design,  the  problem  is  more 
acute  witn  the  one-subject-per-cel 1  design  used 
here  in  which  there  was  an  a  prion  interest  in 
examining  most  of  the  estimable  effects.  It  is 
very  likely  that  a  few  experimental  effects  will 
be  more  than  trivially  biased  by  subject  effects. 
The  subject  factor  (pilot  type)  which  was  included 
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in  the  experiment  did  remove  a  large  amount  of 
subject  variance  and  thus  reduces  this  concern 
somewhat.  Further,  a  ccvariate  task  was  employed 
in  the  experiment  (not  reported  here)  which 
showed  some  relationship  to  the  criterion  task. 

11  Taking  into  account  this  covariate,  re¬ 
sulted  in  only  trivial  effect  estimate  changes 
for  effects  of  interest.  Thus  there  are  grounds 
for  suggesting  that  subject  confounding  was  ade¬ 
quately  controlled,  but  this  problem  was  not 
fully  resolved. 

Second,  there  is  the  issue  of  general iza- 
bility  of  results  because  of  the  in-simulator 
"quasi”  transfer  of  training  design.  Ultimately, 
a  simulator-to-fi^ld  study  with  undergraduate 
Naval  aviators  is  needed  to  confirm  these 
results.  This  study  provides  recommendations  for 
such  an  experiment  and  at  the  same  time  depends 
on  a  field- transfer  study  for  its  own  ultimate 
value.  Confirmatory  results  c^uld  no  a  long  way 
toward  increasing  confidence  in  in-simulator 
results  and  thus  saving  some  of  the  enormous 
expense  associated  with  f ield-transfer  studies. 
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ABSTRACT 


'This  paper  reports  the  status  of  Phase  1  of  an  ongoing  project  to  develop  a 
macro  model  describing  the  decisions  involved  in  developing  training  equipment. 
The  purpose  of  the  model  is  to  assist  managers  in  making  such  decisions  by  providing 
information  concerning  the  tradeoffs  between  cost  and  training  effectiveness  caused 
by  different  configurations  and  choices  of  equipment.  After  the  development  of  a 
preliminary  model,  field  research  was  conducted  to  determine  the  feasibility  of  test¬ 
ing  such  a  model  and  to  collect  information  to  expand  the  preliminary  version  into  a 
more  pragmatic  tool. 

Results  of  the  field  work  led  to  several  conclusions.  First,  many  of  the  types 
types  of  data  needed  to  validate  such  a  model  are  available,  hence  making  such 
a  project  feasible.  Second,  an  examination  of  the  available  data  led  to  an  expansion 
of  the  preliminary  model  to  include  training  value  of  the  various  trainer  characteristics . 
Third,  much  work  is  needed  to  develop  longitudinal  data  bases  of  job  performance 
before  sound  predictions  can  be  made  concerning  the  impact  of  trainer  characteristics 
on  technician  performance  after  graduation. 


PURPOSE 

This  paper  reports  the  status  of  an  ongoing 
project  to  develop  a  macro  model  to  assist 
managers  in  making  decisions  concerning  train¬ 
ing  equipment.  The  model  is  designed  to  permit 
comparisons  of  alternative  equipment  and  con¬ 
figurations,  and  the  associated  cost  and  training 
effectiveness  for  the  alternatives.  The  project 
was  designed  to  meet  the  three  objectives  listed 
below : 

(1)  To  develop  a  macro  model  for  use  in 
making  cost/benefit  tradeoffs  between 
the  various  characteristics  that  may 
be  utilized  in  training  equipment,  and 
to  give  both  military  and  industry 
guidelines  to  justify  decisions  relating 
to  trainer  design. 

(2)  To  determine  the  efficiency  of  a  sam¬ 
ple  of  trainers  currently  in  use. 

(3)  To  determine  the  relative  cost  and 
training  effectiveness  of  various 
characteristics  or  capabilities  that  can 
lie  used  in  training  equipment. 


*  The  authors  would  like  to  express  their 
appreciation  to  Janet  L.  Blanche  for  her 
assistance  during  the  preparation  of  this 
manuscript. 


BACKGROUND 

The  operational  readiness  of  sophisticat¬ 
ed  military  systems  depends,  to  a  greet  extent, 
on  the  skills  of  the  technicians  who  maintain 
those  systems.  As  the  sophistication  of  current 
military  equipment  increases,  the  operational 
training  for  the  use  of  such  equipment  becomes 
simpler.  Conversely,  the  training  of  technicians 
to  maintain  the  equipment  becomes  un  even  more 
complex  task  (1).  In  order  to  assist  in  the 
training  of  maintenance  personnel,  therefore, 
the  Armed  Services  are  increasingly  relying  on 
the  use  of  maintenance  trainers  and  simulators 
(2). 

There  are  two  basic  reasons  underlying 
this  trend.  First  of  all,  it  is  assumed  that 
training  devices  are  capable  of  training  a  stu¬ 
dent  at  least  as  effectively  ns  Instruction  which 
only  utilizes  actual  equipment .  In  fact,  research 
in  instructional  psychology  suggests  that  the  use 
of  actual  equipment  for  training  purposes  may 
be  less  efficient  and  effective  than  the  use  of 
training  devices  which  permit  greater  latitude 
in  material  presentation  (3).  For  example,  a 
training  device  can  be  designed  so  that  an  over¬ 
all  task  is  broken  Into  smaller,  less  complicated 
subtasks  which  can  be  more  readily  grasped  by 
a  stuuent.  In  theory,  this  leuds  to  a  better  even¬ 
tual  mastery  of  the  ‘whole  task  (4).  Training 
devices  also  allow  students  to  practice  and  ob¬ 
serve  procedures  which,  although  necessary  for 
competency,  are  po’entially  too  dangerous  ki 
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terms  of  personnel  and/or  equiprne.it  safety  for 
the  neophyte  to  perform  in  the  training  environ¬ 
ment  (T>).  Finally,  training  devices  are  assumed 
to  provide  inon?  effic ie»it  t  raining  due  to  their 
t>  u  1 1 1  -  in  capabilities  which  allow  automatic  stu¬ 
dent  monitoring,  decreased  instructor  demand 
for  routine  oroblems.  and  increased  student 
experience  and  practice  (8). 

The  second  reason  why  training  devices 
are  becoming  increasingly  popular  is  that  such 
devices  art' often  assumed  to  be  more  cost  offer- 
effective  than  the  use  of  actual  equipment  (7). 
Idle  use  of  actual  equipment  in  training  situa¬ 
tions  is  expensive  in  terms  of  such  factors  as 
fuel  costs,  equipment  wear  and  tear  or  damage, 
and  loss  of  availability  for  operation  whiledivcr- 
ted  to  training  use.  Therefore,  even  when  the 
training  effectiveness  of  a  course  utilizing  a 
training  device*  is  only  equivalent  to  that  of  a 
eourse  utilizing  the  actual  equipment,  if  the 
device  is  less  expensive  to  acquire  and  run 
than  the  actual  equipment,  it  is  a  better  value 
for  training  purposes. 

Although  such  reasons  for  using  training 
devices  are  seductive  in  theory,  these  assump¬ 
tions  have  not  necessarily  been  found  to  be  true 
in  practice.  Therefore,  the  question  that  arises 
is  whether  or  not  training  devices  are  effec¬ 
tive  --either  in  terms  of  cost  or  training 
when  compared  with  more  traditional  training 
methods.  The  principles  of  learning  psychology 
have  been  found  to  fall  short  when  practiced  in 
military  training  situations  (8).  Many  of  the 
"common  sense1’  characteristics  often  included 
in  the  design  of  training  devices  (e.g.  ,  high 
realism)  have  not  been  found  to  be  consistentiv 
effective  (9,10).  Furthermore,  although  the 
primary  goal  of  training  is  to  produce  qualified 
technicians  more  quickly,  only  one  study  has 
been  performed  to  comparatively  evaluate  the 
effectiveness  of  technicians  trained  with  actual 
equipment  versus  those  trained  using  simulators. 
Mo  differences  were  found  between  groups  in 
this  study  (11).  Virtually  no  research  has  been 
conducted  relating  life  cycle  costs  to  training 
effectiveness . 

The  lack  of  hard  facts  concerning  the  use¬ 
fulness  of  various  trainer  characteristics  makes 
tra  ining  device  justification  and  design  tradeoffs 
difficult.  Heeause  of  this,  decisions  are  made 
based  only  on  cost  factor's  since  there  arc  no 
data  available  to  evaluate  training  value.  There¬ 
fore,  both  the  Services'  project  managers  and 
industry  have  been  unable  to  articulate  require¬ 
ments  for  training  devices  in  terms  of  projected 
training  value.  Equipment  for  training  is  often 
proposed,  developed,  and  procured  witiiout  an 
estimate  of  the  expected  training  effectiveness 
and  ‘without  a  thorough  analysts  of  which  tasks 
are  suited  to  the  use  of  a  training  device  or  sim¬ 
ulator.  Such  practices,  however,  mean  that  the 
creation  ofan  optimally  effective  training  device 
is  only  serendipitous . 

Since  effective  training  is  vital  forthedevei- 
opmont  of  the  skilled  technicians  who  ure  n  key 
link  in  the  chain  of  operational  readiness'  (12), 
it  is  necessary  that  incompatibilities  between 
current  assumptions  and  actual  data  be  identified 
and  rectified.  There  is,  therefore,  a  strong 


requirement  that  training  efficiency  and  effec¬ 
tiveness  be  considered  along  with  costs  during 
the  front  end  analysis  and  design  of  new  mainte¬ 
nance  training  equipment.  The  formulation  of 
consistent  and  effective  policies  for  such  designs 
depends  to  a  great  extent  on  the  use  of  cost  and 
training  effectiveness  data  in  a  model  of  logis¬ 
tics,  manpower,  personnel,  and  training.  The 
projections  of  such  a  model  must  be  further 
clarified  and  validated  using  hurd  data  collected 
from  the  field. 

Two  sets  of  factors  must  be  dealt  with  in 
such  a  project:  economic  considerations  and 
training  quality.  Historically,  cost  analysis  has 
been  accomplished  for  tradeoffs  between  actual 
equij  ment  and  simulators  used  for  training.  Cost 
cons  derations  have  been  based  on  investment 
cost  ;  and  efficiency  items  such  as  expected  fuel 
savings.  However,  experience  with  successful 
training  devices  indicates  that  certain  etfici- 
encies  which  can  also  be  assigned  a  dollar 
value  accrue  once  the  trainer  is  fielded.  Exam¬ 
ples  of  these  are  time  saved  in  training,  opera¬ 
tional  equipment  made  available  for  missions 
other  than  training,  lowered  investment  costs , 
reduced  maintenance  costs,  and  less  run  time 
on  operational  equipment.  Such  potential  effi¬ 
ciencies  should  be  identified  and  developed  into 
an  economic  model  for  use  in  prediction  of  life 
cycle  costs  during  front  end  analysis. 

The  second  set  of  factors  which  must  be 
considered  in  such  a  project  deals  with  differ¬ 
ences  in  training  effectiveness  between  courses 
using  training  devices  and  those  using  the  actual 
equipment.  Often  the  assumption  is  incorrectly 
made  that  if  a  training  device  is  less  expensive 
than  the  actual  equipment  as  used  in  training, 
it  is  a  better  buy,  and,  therefore,  should  be 
used.  This  line  of  reasoning  is  specious.  In 
order  to  justify  the  use  of  a  training  device, 
it  is  fallacious  to  consider  only  the  costs  of 
acquiring  and  maintaining  it.  Attention  must 
also  be  focused  on  how  well  each  facilitates 
student  learning,  and  on  the  resulting  benefit 
of  more  quickly  producing  qualified  technicians. 
While  effectiveness  is  not  easily  quantifiable, 
there  are  enough  indicators  to  believe  that,  at 
least  in  gross  terms,  it  can  be  measured  and 
modeled.  Such  information  would  be  helpful  in 
making  sound  management  decisions  on  the  value 
of  simulators  for  training. 


The  evaluation  of  the  cost  and  training  effec¬ 
tiveness  of  maintenance  simulators  and  trainers 
is  not  a  new  idea.  In  past  studies,  investigators 
have  typically  examined  variables  such  as  ac¬ 
quisition  costs  and  training  effectiveness  in 
the  school  Hotting  (13).  Military  training,  how¬ 
ever,  is  intended  to  prepare  personnel  to  per¬ 
form  various  jobs  in  operational  units.  There¬ 
fore,  the  true  measure  of  effectiveness  is  actual 
Job  performance  rather  than  performance  with 
in  a  training  course.  Similarly,  although  com¬ 
parisons  between  acquisition  co3ts  of  training 
devices  versus  other  training  methods  are  an 
important  consideration,  it  is  also  necessary 
to  consider  the  comparative  life  cycle  costs  of 
these  methods  before  an  informed  decision  can 
be  made.  Such  an  examination  must  be  accom- 
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pH  shed  o vl’p  a  broad  spectrum  of  applications 
b v fo r e  a n  u 1 1 i m n t e  criterion  of  training  program 
efl’eeti veness  can  bo  established  or  predicted. 

Two  types  of  tools  are  necessary  in  order 
to  make  sound,  knowledgeable  decisions  during 
the  development  of  maintenance  trainers.  The 
first  of  those  tools  is  a  macro  model  which  can 
be  used  as  a  high  level  filter  in  making  design 
decisions  and  determining  device  justifiability. 
This  type  of  model  would  act  as  a  decision 
screen,  specific  only  enough  for  early  decision 
making  bv  program  managers.  Concomitant  with 
such  a  decision  screen,  a  more  specific,  prag¬ 
matic  design  screen  must  also  be  developed. 
This  type  of  tool  could  be  used  effectively  both 
on  a  lower  level  in  Do D and  in  industry  for  making 
specific  cost /training  effectiveness  tradeoffs  in 
the  design  of  future  training  devices. 

PHASE  I:  FIELD  STUDY 

Phase  I  of  this  project  was  designed  as  a 
pilot  study  to  identify  potential  efficiency  and 
effectiveness  factors,  and  to  develop  a  proto¬ 
type  model  with  them.  Maintenance  trainers 
were  chosen  for  analysis  in  this  phase  of  the 
study  for  the  following  reasons: 

(l)  Cost  indicators  and  potential  savings 
are  not  obvious  as  in  the  case  of  flight 
simulators  where  flying  hours  have  a 
known  cost.  Thus,  the  study  avoids  the 
issues  of  expressing  cost  avoidance  as 
an  efficiency,  and  permits  a  focus  on 
more  subtle  cost  savings  as  well  as 
highlighting  concern  on  effectiveness 
issues . 

(3)  Instances  exist  in  maintenance  train¬ 
ing  where  individuals  in  the  same  skill 
and  weapon  system  were  trained  in  dif¬ 
ferent  ways  --  some  with  actual  equip¬ 
ment  ar.d  others  with  training  devices. 
This  situation  allows  the  collection  of 
control  data  better  suited  for  ?naking 
judgments  of  the  relative  cost  and  train¬ 
ing  effectiveness  of  courses  using  train¬ 
ing  devices  versus  tl’  >se  which  do  not. 

(3)  The  factors  leading  to  the  design  of  an 
effective  maintenance  trainer  are  much 
loss  well  defined  than  for  operator 
trainers.  Therefore,  if  the  character¬ 
istics  leading  to  effective  design  can 
be  determined  and  put  into  a  model  for 
maintenance  trainers,  it  is  probable 
that  atU  opts  to  generalize  such  a 
model  would  be  much  less  difficult  than 
if  the  model  were  first  developed  for 
operator  trainers. 

(4)  Emphasis  on  ways  to  enhance  mainte¬ 
nance  training  to  offset  design  ,  support, 
and  manpower  problems  has  forced  DoD 
to  look  at  ways  to  improve  and  expedite 
support  training.  Selection  of  mainte¬ 
nance  trainers  for  this  work  Is  consis¬ 
tent  with  the  DoD  emphasis  to  improve 
training  for  maintenance. 


Preliminary  Model 

As  discussed  earlier  in  this  paper,  in  or¬ 
der  to  develop  a  valid  tool  to  make  knowledge¬ 
able  decisions  concerning  device  cost  and 
training  effectiveness  tradeoffs,  four  levels  of 
variables  must  be  considered: 

(1)  Training  effectiveness  in  the  school, 

(2)  Training  effectiveness  in  the  field; 

(3)  Acquisition  costs  of  equipment; 

(4)  Life  cycle  costs  of  equipment. 

However,  although  necessary  in  the  creation 
of  such  a  tool,  these  factors  are  not  sufficient 
in  and  of  themselves.  There  are  also  several 
modifying  va riables  which,  although  not  directly 
relating  to  device  effectiveness,  act  as  filters 
integral  to  any  comparison  between  the  effec¬ 
tiveness  of  a  training  device  versus  the  actual 
equipment  (Figure  1).  First  of  all.  the  goals  of 
var’ous  maintenance  training  programs  differ, 
and  the  principles  of  instructional  technology 
do  not  lend  support  to  the  supposition  that  the 
same  type  of  training  will  be  equally  efficient 
for  all  purposes.  (For  example,  if  the  goal  is 
to  teach  the  student  motor  skills  such  as  per¬ 
forming  a  task  requiring“interacti ve  analog  in¬ 
puts,  one  would  probably  not  depend  wholly  on 
a  verbal  explanation  --  without  hands-on  prac¬ 
tice  --  to  teach  such  a  task.)  Therefore,  in 
order  to  compare  the  relative  effectiveness  of 
training  programs,  it  is  important  to  keep  in 
mind  the  objectives  toward  which  the  learning 
situations  are  aimed.  A  second  filter  which 
must  be  considered  in  the  creation  of  such  a 
model  is  the  characteristics  of  the  students 
being  taught.  For  example,  in  teaching  a  low 
level  student  with  vitually  no  experience  o^ 
knowledge  in  the  subject  area  it  would  be  nec¬ 
essary  to  go  into  much  more  detailed  explana¬ 
tions  of  the  subject  matter  than  when  giving 
a  review  course  to  students  who  had  been  per¬ 
forming  the  same  task  for  over  u  year.  A  third 
filter  which  one  must  be  aware  of  in  facing 
such  comparisons  is  the  characteristics  of  the 
instructor  teaching  the  course.  For  example, 
if  an  instructor  does  not  understand  or  like  to 
use  a  particular  technique  for  teaching,  he  or 
she  will  not  utilize  the  method  as  well  as  an 
instructor  who  does. 

Therefore,  although  comparing  the  rela¬ 
tive  effectiveness  of  maintenance  trainer  equip¬ 
ment  and  actual  equipment  training  is  theoreti¬ 
cally  a  straightforward  task,  it  is  important 
to  consider  these  modifying  variables  --program 
goals,  student  experience  and  aptitude,  and  the 
instructor  experience  and  aptitude  --in  order 
to  meaningfully  interpret  any  results  of  such 
a  comparison  (  Figure  1). 

This  framework  was  used  as  a  prelimin¬ 
ary  paradigm  to  describe  the  field  of  training 
device  cost  and  truining  effectiveness  factors. 
The  Phase  I  field  study  was  then  designed  to 
test  the  effect  of  the  independent  variable  of 
training  method  on  the  four  levels  of  device 
effectiveness,  taking  into  account  the  modify¬ 
ing  variables  discussed  above.  The  objectives 


i  A 


407 


Figure  1.  Paradigm  for  Determining  Relative  Effectiveness  of  Two  Training  Methods 


of  this  field  study  were  to  collect  information 
to  develop  this  initial  paradigm  into  a  testable 
model,  and  to  determine  the  feasibility  of  collec¬ 
ting  data  to  test  the  model. 


1.  Safety:  Behaviors  which  show  that  the 
technician  understands  and  follows 
safety  practices  as  specified  in  the 
technical  data; 


Methodology 

To  help  achieve  these  objectives,  two  sets 
of  data  collection  instruments  were  developed. 
I  he  first  of  those  was  a  set  of  Behaviorally 
Anchored  Bating  Seales  ( BARS)  which  were  used 
to  assess  technicians'  performance  in  the  the 
field.  In  order  to  develop  these  scales,  instruc¬ 
tors  in  (lie  role  of  subject  matter  experts  were 
asked  to  create  a  series  of  critical  incidents 
describing  behaviors  which  differentiate  between 
a  good  technician  and  a  poor  one.  These  inci¬ 
dents  focused  on  specific  technician  actions 
closely  related  to  the  job,  and  differentiated 
between  success  and  failure  as  a  maintenance 
technician.  Several  hundred  of  these  incidents 
were  collected,  thematically  analyzed,  and 
placed  into  scales.  They  were  then  rated  by 
the  instructors  on  a  seven-point  scale  with  the 
scale  value  of  1  being  very  poor  performance 
behavior  and  the  scale  value  of  7  being  very 
high  performance  behavior..  Those  incidents 
with  the  lowest  standard  deviations  and  means 
closest  to  l  and  to  7  were  then  placed  on  a 
graphic  type  rating  scale  to  be  used  as  behav¬ 
ioral  anchors  for  the  scale. 

There  are  two  advantages  to  using  BARS. 
Rirst,  the  description  of  the  scale  points  is  writ¬ 
ten  in  terms  that  can  be  easily  understood  by 
the  raters.  Second,  since  the  type  of  person 
who  developed  the  scale  is  also  the  type  of  per¬ 
son  who  uses  the  scale,  the  raters  have  a  vested 
interest  in  using  the  scales  correctly  (15). 

The  use  of  the  BARS  development  techni¬ 
que  in  this  study  yielded  seven  specific  scales: 


2 •  Thoroughness  and  Attention  to  Details: 
Behaviors  which  show  that  the  techni¬ 
cian  is  well  prepared  when  he  arrives 
on  the  job,  carries  out  maintenance 
procedures  completely  and  thoroughly, 
and  recognizes  and  attends  to  symptoms 
of  equipment  damage  or  stress; 

3.  Use  of  Technical  Data:  Behaviors 
which  show  that  the  technician  proper¬ 
ly  uses  technical  data  in  performance 
of  maintenance  functions; 


4. 


System  Understanding:  Behaviors 
which  show  that  the  technician  thor¬ 


oughly  understands  system  operation 
allowing  him  to  recognize,  diagnose, 
and  correct  problems  not  specifically 
covered  in  the  Technical  Orders  and 


publications; 


5  •  Understanding  of  Other  Systems:  Be¬ 
haviors  which  show  that  the  technician 
understands  the  systems  that  are  inter¬ 
connected  with  his  specific  system  and 
can  operate  them  in  accordance  with 
technical  orders; 


6*  Mechanical  Skills;  Behaviors  which 
show  that  the  technician  possesses 
specific  mechanical  skills  acquired 
for  even  the  most  difficult  mainten¬ 
ance  problems;  and _ 


7  •  Attitude;  Behaviors  which  show  that 
the  technician  is  concerned  about  prop¬ 
erly  completing  each  task  efficiently 
and  on  time. 


****** 


1  !ioso  svvon  skills  wort*  found  to  bo  generic. 
Molding'  tr-jo  for  both  systems  utlimately  eho.sen 
for  this  study  phase. 

The  second  data  collection  instrument  was 
a  sorjos  of  questionnaires  for  students,  instruc¬ 
tors,  and  technicians.  Those  questionnaires 
were  designed  to  collect  two  types  of  informa¬ 
tion:  ( t )  demographic  information  on  subject 
background,  training,  and  experience:  and  (2) 
subjective  information  such  as  subjects'  attitudes 
toward  training  devices  in  general,  and  their 
perceptions  and  evaluations  of  the  specific  device 
with  which  they  were  working. 

For  this  phase  of  the  study,  the  F-16  and 
K-3A  AWACS  Navigation  Maintenance  trainers 
were  selected  to  serve  as  the  data  sources  for 
the  development  of  the  preliminary  model  -  These 
trainers  are  two  of  the  most  recent  examples 
of  maintenance  trainers  in  the  field.  The  F-16 
trainer  consists  of  a  series  of  six  freeplay 
systems  designed  to  assist  in  teaching  mainte¬ 
nance  courses  in  the  flight  controls.  Comm/ 
Nav,  electrical  systems,  engine  start,  engine 
diagnostics,  and  engine  run  for  the  F-lfi  air¬ 
craft.  The  AWACS  device  is  a  procedural  train¬ 
er  designed  to  de  used  in  courses  in  navigation¬ 
al  system  maintenance. 

Data  for  this  phase  of  the  study  were  collec¬ 
ted  from  subjects  who  had  used  these  two  framing 
devices,  and  also  from  control  subjects  who  had 
gone  through  the  same  courses  without  using 


the  training  devices.  The  training  effectiveness 
soft  data  were  collected  using  the  three  versions 
of  the  questionnaire  to  guthei  background  data 
concerning  the  subjects  and  their  opinions  of 
training  courses  and  devices .  The  BARS  were 
used  to  determine  the  instructors'  performance 
appraisals  of  those  students  having  just  com¬ 
pleted  the  course  and  of  those  having  previously 
taken  the  courses.  Supervisors’  performance 
appraisals  were  obtained  for  technicians  having 
recently  graduated  the  courses,  and  for  those 
who  had  previously  graduated  the  courses.  This 
rodim  iant  use  of  the.  BARS  was  done  in  order 
to  help  determine  the  validity  of  such  subjective 
judgments,  and  to  partially  ascertain  the  rela¬ 
tionship  between  judgments  of  technician  per¬ 
formance  at  the  school  and  field  levels. 

Hard  data  of  training  effectiveness  were 
collected  through  student  course  test  scores 
and  Work  Unit  Code  (WUC)  information.  WUCs 
yield  hard  data  concerning  group  und  individual 
Level  technician  performance  in  the  field  in  such 
areas  as  removal  time,  repair  time,  retest  OK 
(RKTOK)  rates,  etc. 

Data  concerning  the  acquisition  and  life 
cycle  costs  for  courses  utilizing  and  not  utiliz¬ 
ing  the  trainer  were  also  collected  through  the 
Air  Force  and  the  contractor.  These  field  study 
data  are  summarized  in  Tables  1  and  2.  Miss¬ 
ing  data  in  the  tables  result  from  the  normal 
vicissitudes  of  subject  availability  encountered 
in  a  field  study  of  this  nature. 


fable  1.  Data  Collected  in  Phase  I  Field  Study 
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Revised  Model 

Although  it  was  recognized  during  the  eon- 
st faction  of  the  preliminary  model  that  trainer 
characteristics  are  an  important  variable  direc- 
tly  impact'ng  the  effectiveness  of  training  de¬ 
vices,  a  corrij'-rehonsive  list  01  tiicj.c  charneter- 
istics  could  not  be  developed  until  the  views  of 
the  users  wen1  collected  and  the  impact  of  the 
various  charateristics  on  effectiveness  were 
observed.  This  information  showed  that  bv  and 
larsre  the  greatest  "need"  perceived  by  users 
was  for  high  "realism".  As  stated  earlier  in 
this  paper,  training  literature  does  not  totally 
support  such  a  statement  of  need.  However, 
aftL*r  close  examination  of  the  data,  it  was  found 
that  the  discrepancies  between  the  findings  in 
the  literature  and  the  "universal"  statements 
by  the  users  were  to  a  great  extent  a  result  of 
the  lack  of  a  consistent  definition  for  the  term 
"realism".  For  example,  when  one  instructor 
stated  the  need  for  high  "realism"  in  the  design 
of  an  engine  trim  box  to  teach  a  motor  skill,  he 
was  not  necessarily  referring  to  the  same  de¬ 
vice  cha racteristic  as  an  instructor  asking  for 
high  "realism"  of  engine  sounds.  As  a  result, 
the  characteristic  of  "realism"  was  replaced 
for  the  purposes  of  this  model  by  the  taxonornv 
listed  below. 

1  »  Static  Realism 

a.  Visual  Realism:  The  extent  to 
which  the  device  components  or 
subsystems  uppenr  to  be  the  same 
as  on  the  actual  equipment. 


which  the  device  components  or 
subsystems  are  physically  situ¬ 
ated  as  on  the  actual  equipment. 

*  Auditory  Realism:  The  extent  to 
which  the  device  components  or 
subsystems  approximate  the  sounds 
or  the  actual  equipment. 

d*  Kinesthetic  Realism:  The  extent 
to  which  the  device  components  or 
subsystems  approximate  the  feel 
of  ihe  actual  equipment. 

2  *  Dynamic  Realism 

a.  Temporal  Realism;  The  extent 
to  wnich  the  reaction  time  and  re¬ 
sponse  of  the  device  components, 
or  subsystems  approximate  the 
actual  equipment. 

b*  Extent  of  Simulation:  The  extent 

to  which  the  device  as  a  system 
approximates  the  total  responses 
of  the  actual  equipment  rather  than 
only  following  the  responses  given 
in  the  technical  data. 

The  other  training  device  characteristic 
which  many  users  felt  was  important  to  consider 
was  computer  aided  instruction  (CAH.  This  too 
appeared  to  be  a  taxonomy  rather  than  u  sin  le 

variable.?  65 


1 '  student  Aids:  Those  computer  man¬ 

aged  functions  which  directly  aid  the 
student  to  learn  the  material. 


b.  Spatial  Realism:  The  extent  to 


4  *  Instructor  Aids:  Thone computer  man- 
aj'ed  functions  which  indirectly  aid  the 
instructor  in  touching  the  student  the 
material . 

User  Aids:  Those  computer  manag¬ 

ed  functions  which  facilitate  the  stu¬ 
dent's  use  of  the  training  device. 

Based  on  these  taxonomies  and  the  infor¬ 
mation  collected  in  the  field,  the  preliminary 
model  was  revised  to  expand  the  description 
of  the  parameters  to  be  considered  in  the  design 
and  development  of  a  training  device.  (This  re¬ 
vised  model  is  graphically  depicted  in  Figure 
-- • )  1  be  subjective  data  collected  supported 

the  hypothesis  of  the  influence  of  modifying  var¬ 
iables  on  training  effectiveness.  Of  the  three 
original  modifying  variables  considered,  sup¬ 
port  was  found  to  justify  further  investigation 
ol  the  effects  of  both  student  characteristics 
and  training  program  goals  on  training  effec¬ 
tiveness.  Information  about  these  two  charac¬ 
teristics  consistently  suggested  that  "lower 
level  students  -  -where  this  is  defined  bv  skill, 
knowledge,  or  experience  -can  best  be' taught 
using  different  techniques  than  those  used  for 
'higher  level"  students.  Similarly,  the  data 
suggested  that  different  sets  of  device  charac¬ 
teristics  would  best  be  utilized  in  teaching  dif¬ 
ferent  program  goals  (e.g.  .  mechanical  skills 
or  theoretical  troubleshooting). 

The  attitudes  of  an  instructor  toward  a 
training  device,  however,  (although  logically 
related  to  how  well  the  device  is  used  by  the- 
instructor  and  how  well  the  student  uses  and 
learns  from  the  device)  were  not  found  to  sig- 
ni  fie  ally  impact  training  effectiveness  in  the 
sites  and  programs  investigated  in  Phase  I. 
In  the  two  programs  studied,  the  instructors 
as  a  whole  were  rather  ambivalant  towards  the 
devices  when  they  first  began  to  use  them,  but 
they  became  more  positive  as  their  experience 
with  the  devices  increased.  Although  this  may 
cause  differential  effects  in  training  between 
students  taught  directly  after  trainer  acquisi¬ 
tion  as  opposed  to  those  taught  later,  the  effects 
of  this  variable  appear  to  be  equated  over  a  per¬ 
iod  time.  A  lesson  to  be  learned  here,  however, 
is  that  improved  instructor  training  immediate¬ 
ly  following  the  fielding  of  the  training  device 
can  positively  impact  the  acceptance  and  use 
of  the  device. 

Although  the  modifying  variables  mention¬ 
ed  above  have  significant  impact  on  the  quality 
of  training  and  must  bo  considered  in  the  inter¬ 
pretation  of  effectiveness  data  ,  the  characteris¬ 
tics  of  the  device  itself  are  of  superordinate 
importance  in  their  effect  on  cost  and  training 
effectiveness ,  For  the  purpose  of  this  model, 
‘'effectiveness"  is  defined  as  a  relative  term. 
Since  training  effectiveness  cannot  currently 
be  judged  on  an  absolute  scale,  i!  must  be 
examined  through  a  comparison  between  the 
quality  of  technicians  in  the  field  who  have  been 
taught  using  a  trair  ng  device  versus  those  who 
have  not.  The  relationship  of  these  character¬ 
istics  to  the  overall  model  is  represented  to 
the  horizontal  "slice"  depicted  in  Figure  2.  In 
Figured,  one  of  these  "slices"  has  been  rotated 
5)0  degrees  in  order' to  give  a  clearer  view  ofits 
components .  t 


At  this  level,  me  model  can  potentially  aid 
in  making  more  prugmatic  decisions  concern¬ 
ing  the  design  of  training  devices.  Given  u  sound 
definition  of  training  device  cost  (us  defined 
both  by  acquisition  and  life  cycle  costs),  and 
the  establishment  of  a  sufficient  data  buse,  it 
will  be  relatively  easy  to  determine  the  cost 
associated  with  different  training  device  de¬ 
signs  under  consideration.  For  ’  example  ,  in 
low  quantities ,  those  characteristics  defining 
static  realism  are  relatively  low  in  cost  as 
opposed  to  those  characteristics  defining  dynam¬ 
ic  realism.  The  missing  factor  in  the  design 
and  development  considerations  of  today,  how¬ 
ever,  is  the  concomitant  consideration  of  com¬ 
parative  training  effectiveness .  By  developing 
a  large  enough  data  base,  it  will  eventually 
be  possible  to  generate  a  series  of  equations 
which  will  allow  managers  to  make  decisions 
as  to  the  relative  training  effectiveness  of  var¬ 
ious  alternative  design  configurations  under 
consideration.  It  will  then  be  possible  to  deter¬ 
mine  not  only  the  cost  of  acquiring  and  using  a 
training  device  ,  but  also  the  training  effective¬ 
ness  of  such  r.  device  relative  to  the  actual 
equipment.  For  example,  in  Figure  3,  for  a 
certain  training  goal  and  student  level  there 
are  hypothetically  two  different  configurations 
which  could  be  used  in  the  design  and  develop¬ 
ment  of  a  trainer,  both  of  which  would  be 
equivalent  to  each  other  and  to  the  actual  equip¬ 
ment  in  terms  of  training  value.  The  recom¬ 
mendation  would  then  be  to  either  use  the 
more  cost  effective  of  the  two  alternatives, 
or  to  increase  the  projected  level  of  training 
effectiveness  by  selecting- a  device  configura¬ 
tion  higher  on  the  V-axi  s  . 

The  development  of  this  paradigm  into  a 
useable  tool  for  managers  depends  upon  the 
creation  of  performance  measures  to  determine 
the  training  value  of  various  design  character¬ 
istics.  Once  this  is  accomplished,  tradeoffs 
between  cost  and  training  effectiveness  can  be 
determined.  It  should  be  recognized  that  the 
design  characteristics  for  a  training  device 
or  simulator  are  the  driving  factors  in  effec¬ 
tiveness  potential  and  are  also  the  major  deter¬ 
minants  of  the  utlimate  cost.  This  leuds  to  the 
conclusion  that  cost  and  benefit  tradeoffs  can 
be  made  for  design  characteristics  once  the 
manager  determines  specific  training  objec¬ 
tives.  It  will  then  be  possible  to  approach 
the  larger  question  of  comparing  cost  and 
training  effectiveness  in  order  to  make  a  ration¬ 
al  choice  between  equipment  alternatives. 

P BASICS  11  AND  III:  MODEL 
REFINEMENT  AND  VALIDATION 

The  model  as  described  above  was  develop¬ 
ed  using  Lvo  maintenance  trainers.  Before  it 
can  be  considered  as  a  viable  instrument  which 
can  be  applied  to  other  maintenance  training  de¬ 
vices  It  must  undergo  validation  and  refinement . 
To  do  this,  it  must  be  ascertained  whether  the 
parameters  of  the  model  vulidly  apply  to  other 
maintenance  training  device  systems,  whether 
the  indicators  of  cost  and  training  effectiveness 
are  meaningful  for  the  evaluation  of  mainte¬ 
nance  training  devices,  and  whether  the  design 
characteristics  included  in  the  model  signifi¬ 
cantly  impact  the  usefulness  oT  the  device  in 
terms  of  these  indicators. 


411 


TRAINING  EFFECTIVENESS 

(RELATIVE  TO  ACTUAL  EQUIPMENT) 


412 


'A 


gi 

Z  a. 

^5 


£  o 


o  -) 

uj  < 
u.  D 
U.  H 
Ui  O 


o 


Z  £ 


p 


CONFIGURATION  1 

CONFIGURATION  2 

VISUAL  REALISM 

STATIC 

SPATIAL  REALISM 

> _ 

REALISM 

AUDITORY  REALISM 

r 

KINESTHETIC  REALISM 

DYNAMIC 

TEMPORAL  REALISM 

REALISM 

[extent  OF  simulation 

INSTRUCTOR  AIOS 

COMPUTER 

AIDS 

STUQENT  AIOS 

J 

> 

USER  AIOS 

/ 

LEVEL  OF  CHARACTERISTIC 

LEVEL  OF  CHARACTERISTIC 

F  igutt  3.  OvMitaJ  Vi»w  of  R«v«*d  Mod#! 


413 


Although  this  model  has  intuitive  appeal, 
it  cannot  be  used  until  it  is  validated.  In  the 
future,  it  will  be  necessary  to  collect  longitud¬ 
inal  data  along  various  points  of  the  charactor- 
ist.cs  continua  and  relate  these  to  both  cost 
and  training  effectiveness.  Such  a  validation 
must  be  longitudinal,  to  say  the  least,  and  is 
outside  tin*  scope  of  tin*  present  study.  Valida¬ 
tion  at  this  point  in  time  will  consist  of  collect¬ 
ing  comprehensive  data  on  the  training  systems 
currently  in  the  field.  The  systems  will  be  rated 
rated  and  placed  on  the  characteristics  continua, 
and  used  as  inputs  for  the  model  validation  and 
refinement. 

The  next  step  will  be  to  collect  data  from 
courses  using  maintenance  training  devices  and 
their  concomitant  control  courses.  These  data 
will  be  analyzed  in  terms  of  the  structure  oi 
the  model  to  test  for  goodness  of  fit.  The  model 
will  then  be  refined  in  order  to  better  account 
for  all  data  based  on  any  mismatches  in  the 
comparison.  Once  the  model  has  been  refined 
in  Phase  11,  it  will  be  further  validated  using 
a  broader  spectrum  of  training  devices  and 
simulators.  Much  is  already  known  about  re¬ 
quirements  for-  development  and  design  of 
an  effective  operator  trainer.  Phases  1  and  11 
of  the  current  study  will  offer  significant  parallel 
data  for  maintenance  trainers .  This  information 
will  he  synthesized  and  validated  for  the  crea¬ 
tion  of  a  generic  model  in  Phase  111. 

Several  types  of  questions  a  re  to  be  answer¬ 
ed  in  Phase  III.  If  the  indicators  of  cost  and 
training  effectiveness  are  found  to  b»*  valid  tor 
maintenance  trainers,  are  they  equally  valid 
for  operator  and  crew  trainers?  Do  those  indi¬ 
cators  apply  equally  well  to  part  task  training 
devices  and  to  system  devices?  Those  kinds 
of  questions  must  be  answered  overtime  before 
the  model  can  be  thought  of  as  a  univeral  tool 
to  be  used  by  management. 

During  the  Phase  111  validation .  each  ser¬ 
vice  will  be  asked  to  apply  the  model  throughout 
development  and  acquisition  of  an  emerging  sys¬ 
tem.  Use  of  the  model  during  this  phase  will 
be  controlled  only  to  the  extent  that  each  ele¬ 
ment  of  the  model  must  he  investigated.  (Con¬ 
ditions  may  exist  where  one  or  more  ot  the 
elemeat.s  may  not  apply  to  the  chosen  system. 
It  is  important,  however,  that  as  much  infor¬ 
mation  as  possible  be  collected  concerning  all 
parameters  • ) 

The  prediction  of  performance  data  will 
clearly  he  the  most  difficult  to  validate.  This 
is  compounded  by  the  fact  that  not  enough  atten¬ 
tion  has  previously  been  given  to  tin*  systema¬ 
tic  gathering  of  such  i  (.formation.  \s  a  result, 
there  is  a  lack  of  a  sufficient  data  base,  dob 
performance  data  for  the  two  systems  investig¬ 
ated  in  this  study  were  limited  to  a  compari¬ 
son  of  Wl’C  items,  time  to  repair.  KliTOK 
rates,  etc.  Other  performance  measures  were 
subjective.  Sine**  the  purpose  of  e  Electing 
these  data  was  to  establish  their*  usefulness  as 
indicators  of  effectiveness  rather  than  to  eva¬ 
luate  trainers,  their  use  for  this  phase  was 
acceptable .  In  the  validation?  phase,  however, 
it  is  reasonable  to  estimate  probable  effective¬ 
ness  gams  on  emerging  systems  from  using 


performance  data  from  existing  systems.  Ihesc 
estimates  could  then  serve  as  a  basis  for  selec¬ 
ting  design  features  and  justifying  design 
decisions . 

DISCUSSION 

The  model  presented  above  was  prepared 
as  an  archetype,  showing  examples  of  the  types 
of  considerations  which  need  to  he  made  in  the 
creation  of  training  devices.  Such  considera¬ 
tions  will  aid  in  making  knowledgeable  decisions 
relative  to  cost  and  training  effectiveness  dur¬ 
ing  device  development  and  design.  AJtnough  the 
model  is  empirically  based  and  analytically  de¬ 
veloped,  it  is  not  without  shortcomings  which 
primarily  reflect  the  state  of  naintr  nance  train¬ 
ing  the  Armed  Services  tod. 

The  concensus  of  field  ersonnel  is  that 
training  courses  such  as  t  u*  being  studied 
in  this  projectare  ofinsignif  nt  value  to  train¬ 
ing  except  for  use  as  a  gener  verview.  Kather*. 
many  people  feel  tnat  the  rnos  ilnable  training 
is  done  during  on-the-job  training  (OIT).  This 
assumption  is  partially  supported  by  the  pre¬ 
vious  literature  in  which  no  differences  are 
found  between  the  performance  of  students  hav¬ 
ing  been  through  courses  using  actual  equip  nent 
and  those  who  went  through  course's  using  a  train¬ 
ing  device.  As  stated  lr  the  introduction  to  this 
paper,  «me  of  the  shortcomings  of  this  research 
is  that  the  focus  has  been  primarily  on  effective¬ 
ness  vis  a  vis  the  school  situation  rather  than 
the  actual  field  situation.  However,  the  very 
structure  of  maintenance  training  in  the  Ser¬ 
vices  supports  this  assumption:  students  are 
taught  basic  information  in  the  school  and  more 
specific  information  through  either  formal  or* 
informal  OJT  upon  graduation.  Therefore, 
all  technicians  will  eventually  reach  a  minimum 
standard  of  qualification  for  the  job:  if  not.  they 
will  he  moved  into  a  different  job  classification. 
As  a  result,  the  question  is  not  how  effective 
is  the  training,  but  rather  how  quickly  dues 
the  student  reach  the  level  of  minumum  qualifi¬ 
cation  (  Figure  4). 


Figure  4.  Completive  Rates  with  Which  Various  Training  Courses  Prepare 
Students  to  Meet  Minimum  Standard  of  Performance 


Such  criterion- referenced  measures  of 
training  effectiveness  may  no  longer*  be  appro¬ 
priate  since  the  advent  of  training  devices.  It 
is  not  safe  to  assume  that  because  a  training 
course  has  been  taught  in  a  classroom  using 
actual  equipment  as  an  adjunct,  that  a  training 
device  must  be  designed  to  fit  this  mode.  This, 
coupled  with  the  fact  that  there  have  been  vast 
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advances  in  instructional  technology,  suggest 
that  now  is  a  good  time  to  reevaluate  cur¬ 
rent  training  methods  ,  and  devise  a  system  which 
will  enhance  effectiveness  in  an  absolute,  not 
just  a  relative,  sense.  It  is  not  necessarily 
logical  to  take  the  same  course  documentation, 
as  is  currently  done,  and  exchange  the  terms 
"training  device"  for  "actual  equipment." 

There  is  another  difficulty  which  hinders 
the  development  of  devices  which  are  more 
training  effective  than  the  actual  equipment. 
Although  there  have  unquestionably  been  ad¬ 
vances  in  the  area  of  instruetioaal  technology 
over  the  past  ten  years,  in  the  main  there  is 
no  empirical  evidence  to  support  their  effec¬ 
tiveness.  Researchers  have  been  pleading  for 
more  background  investigation  into  these  areas. 
However,  although  virtually  every  technical  pa¬ 
per  on  the  topic  seems  to  end  with  a  statement 
of  this  need,  the  research  itself  is  being  done 
slowly.  If  we  truly  wish  to  improve  the  quality 
of  maintenance  technicians  --  and  concomitant¬ 
ly  the  operational  readiness  of  the  Armed  Ser¬ 
vices  --  it  is  well  past  the  time  to  earnestly 
investigate  these  matters.  More  comprehensive 
performance  data  must  he  collected,  and  a  long¬ 
itudinal  data  base  developed  so  that  meaningful 
judgments  of  effectiveness  can  be  made-. 

Some  of  the  reasons  for  the  lack  of  empir¬ 
ical  evidence  supporting  the  worth  of  training 
devices  vis  a  vis  effectiveness  have  been  dis¬ 
cussed  in  this  paper.  However,  the  lack  of 
such  evidence  is  not  surprising:  course  approach 
does  not  change  with  the  addition  of  a  trainer; 
hotter  instructional  technology  is  not  implement¬ 
ed  in  trainer  design;  the  usefulness  of  the  vari¬ 
ous  instructional  technologies  is  not  known  and 
is  not  being  researched.  Training  devices  cur¬ 
rently  represent  an  engineering  solution  to  a 
nonengineering  problem.  This  must  be 
changed.  Creative  designs  and  solutions  in 
training  terms  must  be  applied  in  the  develop¬ 
ment  of  such  devices,  with  the  application  of 
new  techniques  to  this  new  technology. 
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The  34th  Tactical  Airlift  Training  Group  (TATG)  at  Little  Rock  AFB  provides  initial  and 
tactical  mission  qualification  training  to  C-130  crewmembers.  One  of  the  new  features  of  the 
newly  delivered  simulators  is  the  inclusion  of  Stationkeepinq  Equipment  (SKE).  A  complex  set 
of  procedures  for  proper  utilization  of  SKE  during  'formation  airdrops  forms  a  large  portion 
of.vthe  tactical  mission  qualification  traininq  course. "s-The  Training  Programs  Branch  of  the 
3<T  TATG  conducted  a  study  to  explore  the  application  of  th^JFSy  to  pilot  and  navigator 
training.  The  study  was  conducted  using  four  test  classes.  After  a  standard  academic  course, 
classes  of  pilots  and  navigators  were  divided  into  test  and  control  groups.  The  test  groups 
were  trained  using  a  ore-designed  simulator  syllabus  and  their  performance  was  measured  in 
the  aircraft.  The  control  groups  received  their  traininq  only  in  the  a-^craft  before 
completing  the  same  performance  measurement.  The  study  results  in  terms  of  subjective  and 
objective  data  showed  that  the  IFS  could  reasonably  support  a  traininq  effectiveness  ratio  of 
approximately  0.5.  The  best  training  strategy  appears  to  be  an  integration  of  IFS  missions 
amonq  flying  missions  and  ground  training  rather  than  in  one  blocks  We  recommend  inclusion 
of  the  IFS  in  mission  qualification  traininq.  We  also  recommend  a  re-evaluation  of  the 
mechanics  of  the  proficiency  advancement  concept. 


INTRODUCTION 

Background :  The  34th  Tactical  Airl.tt  Traininq 
Group  at  Little  Rock  AFB  is  tasked  to  provide 
the  000  C-130  traininq  to  student  pilots  and 
navigators.  Mission  qualification  traininq 
covers  the  areas  of  airdrop,  formation,  and 
shortfield  operations.  Student  pilots  and 
navigators  learn  formation  procedures  for  both 
instrument  and  visual  operations.  Formations  in 
instrument  conditions  rely  on  Stationkeepinq 
Equipment  (SKE)  to  maintain  aircraft 
separation.  Traininq  SKE  procedures  makes  up  a 
large  proportion  of  the  curriculum. 

Stationkeepinq  Equipment:  The  Inlraf ormation 

Pos’itionfnq  i^et  AN/APN  ~1&9A  is  a  system  which 
allows  up  to  36  aircraft  to  maintain  fixed 
separation  between  airplanes  in  formation,  and 
tc  locate  and  identify  e  ch  other  during  day  and 
riqft  flights  under  instrument  conditions. 
Oper  ‘lion  of  this  equipment  and  specialized 
procedures  for  formation  flight  involving  pilots 
and  navigators  are  the  main  a**eas  of  concern  in 
this  Study. 

Instrument  Flight  Simulator  ( I F_b ) :  In  November 
m  the  first  1-1  TOT  rTTglit  simulator,  type 
A/F37-AT59,  was  shipped  from  the  Singer-Link 
Corporation,  Binghamton,  NY  to  little  Rock  AFb. 
This  technologically  advanced  aircrew  training 
device  (Ain)  provides  a  training  environment 
us{ng  a  simulated  l-13=J  aircraft  cockpit.  Th*> 
cockpit  can  provide  simultaneous  training  tor  a 


pilot,  a  copilot,  a  flight  enqineer  and  a 
navigator,  controlled  from  an  onboard, 
console-type,  operators’  station.  The  IFS  is 
equipped  with  a  six  degree  of  freedom  motion 
base  which  can  provide  hiqhly  realistic  motion 
cues.  A  list  of  some  of  the  simulator's  other 
design  capabilities  includes  full  SKE  airdrop 
simulation,  radar  simulation,  manual  or 
pre-programmed  malfunctions,  a  library  of 
demonstrations  of  typical  maneuvers  such  as 
instrument  approaches  or  airdrop  procedures,  and 
an  emergency  procedures  monitor  function. 

Acceptance  test  procedures  on  the  .;FS  were 
completed  at  Little  Rock  AFB  in  April  1981.  The 
USAF  Airlift  Center  Interim  Report  presents 
conclusions  concerning  IFS  capabilities  based  on 
Qualification  Operational  Test  and  Evaluation 
(IjOT&E)  results  (1).  Some  of  the  conclusions 
were:  1.  T fie  new  simulator  is  superior  to  its 
1:0  year  old  predecessor.  /.  The  new  simulator 
is  capable  of  training  crew  tasks  that  cvmot  be 
accomplished  in  the  old  simulator.  3.  The 
navigator  station  enhances  navigator  training 
and  crew  coord inat ion.  Annex  l  of  that  QOl&E 
lists  special  aircrew  tasks  that  were  found  *o 
be  trainable,  partially  trainable,  o*‘  not 
trainable  in  the  IFS.  Vnortg  those  tasks  listed 
as  trainable  were:  Stat ionkeeping  Equipment 
checklist  SKE  formation  escape,  Sm  formation 
recovery,  airdrop  checklist,  and  airborne  radar 
approaches.  This  information  suggested  that  the 
new  simulator  wouid  have  direct  application  to 
pilot  and  navigator  mission  gual it icat ion 
training. 
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Summary  of  Primary  Objectives:  In  June  1931, 
the  Training  Programs  Branch  of  34th  TATG 
initiated  a  study  to  explore  possible 
application  of  the  IFS  in  pilot  and  navigator 
mission  qualification  training.  Listed  below 
are  the  four  areas  of  principal  concern  in  this 
study  arranged  in  order  of  decreasing  importance. 

1.  The  primary  objective  of  this  study  was 
to  examine  the  transfer  of  training  using  this 
device.  Positive  transfer  implies  that  as  a 
result  of  training  in  the  simulator,  less  time 
is  needed  in  the  aircraft  in  order  to  attain  a 
predetermined  performance  criterirn. (2,  3)  2. 
A  second  objective  was  to  investigate  possible 
course  structures  to  optimize  simulator 
effectiveness.  Also  under  consideration  was  the 
best  arrangement  of  the  course  from  the  point  of 
view  of  efficient  scheduling.  3.  The  third 
objective  was  to  produce  and  prove  simulator 
courseware  and  job  performance  aids.  Since  SKE 
procedures  had  never  before  been  presented  in  a 
simulator  there  was  no  applicable  courseware  in 
existence.  4.  An  additional  objective  of  the 
study  was  to  determine  the  efficacy  of  the 
instructor  traininq  program.  A  further  task  was 
to  establish  a  qualified  force  of  instructor 
pilots  and  naviqators  in  adequate  numbers  for 
the  study. 

METHOD 

A  number  of  types  of  training  effectiveness 
studies  were  considered.  The  transfer  of 
traininq  desiqn  was  determined  to  be  the  most 
appropriate  to  determine  whether  ATD  training 
would  improve  a  student's  subsequent  performance 
in  the  aircraft  (4). 

Study  Desi  qn:  Throuqh  the  use  of  a  transfer  of 
training  desiqn,  experimental  and  control  groups 
would  be  evaluated  both  objectively  and 
subjectively.  The  methodoloqy  of  data  gathering 
required  that  experimental  groups  be  exposed  to 
a  pre-desiqned  simulator  syllabus  and  then  have 
their  performance  measured  in  the  aircraft.  The 
control  group  would  receive  their  training  only 
in  the  aircraft  before  completing  the 
performance  measurement.  There  were  many 
constraints  placed  upon  the  implementation  of 
this  experimental  program. 

Time  and  Numbers:  One  of  tne  most  profound 
limitations  was  ~the  availability  of  simulator 
time.  In  July,  utilization  reached  48  hours 
divided  into  four,  \2  hour  days.  All  use  of  the 
simulator  was  lost  after  1  Oct  81  due  to 
installation  of  a  visual  display  system.  Based 
on  this  limited  simulator  avai 1 abi : ■ ty  four 
student  crews  each  from  classes  81-OK,  81-014, 
S i -01b,  and  81-018  were  selected  as  the  test 
group.  The  remaining  students  in  these  classes 
and  the  student  population  in  the  intervening 
odd  numbered  classes  made  up  the  control  group. 
The  student  test  group  was  composed  of  30  pilots 
and  IS  naviqators.  While  it  would  have  been 
desirable  to  get  a  more  statistically 
Significant  sample  for  the  test  group,  the 
actual  number  of  subjects  was  thus  limited. 


Missions:  Numerous  studies  have  shown  that 
a  traininq  effectiveness  ratio  of  .48  is  a  good 
average  value  (5).  This  value  was  used  as  a 
starting  point  from  which  planning  proceeded. 
Based  on  simul:tor  time  available,  frequency  of 
class  starts,  and  the  number  of  days  allocated 
for  flying  training,  a  preliminary  decision  was 
made  to  produce  a  traininq  block  of  four 
simulator  missions.  Existinq  instructional 

plans  were  changed  only  in  the  flying  phase  of 
instruction  for  both  p.lots  and  navigators.  For 
the  test  qroup,  this  chanqe  included  four 
simulator  missions  and  six  flyinq  missions 
followed  by  the  standard  evaluation.  The 
planned  sequence  after  academics  became  one 
flying  mission,  a  block  of  four  simulator 
missions,  fve  more  flying  missions  and  an 
evaluation. 

The  simulator  was  designed  to  allow  for 
preprogramming  of  mission  profiles.  Due  to 
system  limitations  dur;ng  IFS  testing,  this 
feature  was  not  available  for  this  study. 

Subjects:  The  first  class  was  conducted 

with  students  handpicked  on  the  basis  of 
previous  C-130  experience  or  strong  performance 
in  the  initial  qualification  course.  With  this 
background,  they  would  not  be  hurt  by  any 
shortcomings  in  the  initial  syllabus  (6,  7). 

Also,  the  best  qualified  students  were  expected 
to  point  out  weak  areas  in  the  program. 

Students  from  the  next  ’.hree  classes  were  chosen 
so  that  the  test  group  would  closely  approximate 
the  control  qroup  in  experience  and  aptitude. 

Data:  The  data  collected  for  this  study  fell 
into  one  of  two  categories,  subjective  or 
objective. 

Subjective  Data:  The  instructor  mission 

reports  provided  course  developers  with  their 
first  feedback  for  improving  the  course  as 

traininq  progressed.  This  report  aided 
oevelopers  in  resolvinq  student  critique  items. 

Students  were  asked  to  complete  a  critique 
of  tne  simulator  course  before  and  after  the 
flying  phase  of  traininq.  The  critiques  used  a 
1  to  5  numerical  qrading  scale  to  rate 

approximately  18  course-related  areas  with  room 
for  comments  and  student  data.  In  this  way, 
student  attitudes  could  be  gauged  before  flight 
traininq  to  qet  immediate  feedback  on  the 
details  of  the  simulator  curriculum.  The 
critique  administered  after  the  flying  phase  was 
intended  to  indicate  the  student  perceptions  of 
simulator  realism  and  how  well  it  prepared  them 
for  their  aircraft  missions  from  the  perspective 
of  course  completion. 

In  order  to  get  the  instructors'  overall 
view  of  the  course,  a  meeting  was  held  on  30  Oct 
81  after  all  simulator  training  ^as  completed. 
The  instructors  had  had  enough  time  to  mull  o.er 
the  proqram  by  the  time  the  comments  in  the 
mission  reports  were  raised  for  gereral 
discussion.  A  consensus  was  reached  in  each 
case  about  the  validity  and  relative  importance 
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of  each  item  reported.  This  also  provided  a 
trigqer  for  further  discussion  on  several 
topics.  This  meeting  was  the  last  source  of 
subjective  data  considered. 

Objective  Data:  Numerical  data  is  extracted 
from  student  training  records  and  evaluation 
worksheets.  Instructors  assign  performance  and 
knowledge  grades  based  on  student  proficiency. 
The  grading  system  spans  the  numerical  grades  1 
to  4  for  performance  and  A  to  D  for  knowledge 
levels.  Data  on  flying  time  and  number  of 
sorties  completed  can  also  be  obtained  from 
these  forms. 

Courseware:  Instructor  quides  and  student  study 
guides  are  routinely  distributed  as  a  part  of 
courseware  for  the  academic  and  flying  phases  of 
traininq.  With  the  addition  of  simulator 
missions,  additional  quides  were  developed  for 
both  the  instructors  and  the  students.  These 
test  guides  were  developed  with  two  objectives 
in  mind:  the  need  to  prepare  students  to  use 
the  simulator  time  effectively,  and  the  need  to 
adequately  prepare  instructors  for  the  unique 
simulator  traininq  mission. 

PROCEDURES 


The  test  program  will  be  considered  in  terms  of 
two  major  phases:  Design  and  Implementation. 
Also  considered  here  will  be  the  problems 
encountered  throughout  the  study. 

Design  Phase:  Course  developers  were  first 
exposed  to  the  IPS  in  Auq  ou  on  a  Training  Gr'uup 
sponsored  trip  to  the  Sinqer-Link  plant  at 
Binqhamton,  NY.  Durinq  the  visit,  Singer-Link 
personnel  demonstrated  the  capabilities  and 
desian  charater istics  of  the  cockpit  simulator, 
the  instructor  onboard  station,  and  the  motion 
base.  The  developers  returned  after  four  days 
with  enough  data  to  prepare  a  preliminary 
planning  document  in  Oct  80  that  outlined 
assumptions,  a  scenario  for  incorporat inq  the 
simulator  in  traininq,  a  study  proposal  and 
possible  mission  profiles. 

After  the  simulator  arrived,  it  became 

apparent  that  the  test  profiles  in  the  computers 
were  unsuitable  for  traininq.  It  also  became 
apparent  tnat  neither  the  equipment  nor 

qualified  personnel  were  available  fo  reprogram 
the  memory  discs  in  the  computer.  ,his  'act  had 
a  major  ef  fect  on  courseware  pi  inn inq  and  the 
instructor  qualification  program  since  the 

developers  had  hoped  to  train  the  instructors  on 
the  mission  profiles  to  be  later  used  with 

students. 


Implementat ion _ Phase : 

qua! if Teat ion  students 
simulator  training  on 
student  crews  completed 
Oct  81  and  completed  the 


The  first  mission 
(class  81 -Jl?)  beqan 
<*3  Jun  81.  The  last 
simulator  traininq  on  1 
course  on  ,’t»  Oct  81. 


Between  each  of  the  classes,  course 
developers  made  revisions  to  students  materials 
as  required.  Course  developers  consolidated, 
printed,  and  distributed  additional  techniques 
covering  simulator  operation  and  failure 
trends.  Training  time  was  saved  as  the 
instructors  were  made  aware  of  the  experiences 
that  preceding  instructors  documented  on  the 
mission  reports.  In  general  all  possible  aids 
were  provided  and  improved  throughout  the 
program  to  insure  peak  efficiency  of  student 
training  and  maximum  instructor  acceptance  of 
the  program. 

All  students  considered  in  this  study 
attended  the  standard  academic  training 
program.  For  pilots,  this  meant  five  days  of 
academics  followed  by  an  initial  flying  mission 
and  then  a  final  academic  day.  For  the 
navigators,  there  were  eight  days  in  academics 
and  then  the  first  flight  mission.  Following 
academics,  the  students  who  made  up  the  control 
group  flew  a  scheduled  program  of  eight  missions 
and  a  flight  evaluation.  The  actual  number  of 
missions  an  individual  completed  varied. 

Upor  completion  of  academics,  the  simulator 
group  began  a  block  of  four  simulator  missions 
of  four  hours  each.  The  actual  number  of  days 
needed  to  complete  this  training  varied  due  to 
simulator  availability.  There  was  some 
flexibility  built  into  the  profiles  to  allow  the 
instructors  to  concentrate  training  on 
individual  student  weaknesses. 

At  the  end  of  the  simulator  phase,  the 
simulafor  students  flew  a  program  of  six  flying 
missions  and  a  checkride.  The  test  program 
called  for  these  traininq  flights  to  concentrate 
on  low  level  visual  procedures.  To  accomplish 
this,  the  test  group  and  the  control  group  would 
fly  in  separate  formations  during  the  flying 
phase.  After  the  test  group's  fourth  flying 
mission,  the  two  groups  could  be  integrated  into 
the  same  formation  in  order  that  the  test  group 
fly  in  SKE  formations  as  a  review  prior  to  their 
checkrides.  This  plan  proved  larqelv  unwrrkable 
for  operational  reasons,  so  the  flight  mission 
profiles  flown  were  essentially  the  same  for 
both  qroups. 

Problems  Encountered:  Other  than  the  resistance 
to  chanqe  anticipated  in  a  significant 
alteration  of  traininq,  the  following  major 
problem  areas  were  encountered  durinq  the  test 
program:  simulator  maintenance,  instructor 
attrition,  and  scheduling  as  it  affected  profile 
changes  and  proficiency  advancement.  These 
problems  existed  during  the  entire  test  program, 
and  although  they  were  overcome  to  the  extent 
that  the  proqram  was  completed,  they  will 
continue  to  impact  any  full-scale  incorporat ion 
of  the  simulator  into  mission  qualification 
traininq.  The  background  and  impact  of  the 
problem  areas  will  be  considered  here;  possible 
solutions  can  b**  found  in  the  I’ecommendat  ions 
section. 
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Simulator  Maintenance:  The  most  obvious 
problem  that  arises  when  trying  to  create  a  new 
training  syllabus,  simultaneously  with 
full-scale  development  of  an  ATO,  is  building  a 
core  of  knowTedge  about  the  device.  In  the  case 
of  IF$  maintenance,  this  was  particularly  true. 
The  manning  ipvel  had  ueen  fixed  at  a  number  of 
personnel  to  maintain  the  four  old  analog 
simulators  plus  two  new  devices.  During  the 
study  period,  this  manning  was  required  to 
maintain  the  four  old  simulators,  the  rew  IFS, 
and  two  Cockpit  Procedures  Trainers. 
Additionally  there  was  a  requirement  to  retrain 
maintenance  personnel  from  analog  to  diqital 
loqic. 

One  area  of  maintenance  particularly 
affected  by  manning  and  training  was  simulator 
software.  Durinq  the  study  period  there  were 
literally  hundreds  of  software  deficiencies  the 
status  of  which  was  still  unresolved  between 
Sinqer-Link,  Aeronautical  Systems  Division 
(ASD),  and  the  Data  Base  Engineering  Prototype 
Site  (OEPS)  personnel.  Due  to  the  manning 
limitations  and  training  level  of  those  assiqned 
to  DEPS,  very  few  discrepancies  were  corrected 
during  the  study  period.  Instructors  were 
forced  to  ignore  and  train  around  the  vast 
majority  of  software  errors. 

Instructor  Attrition:  In  order  that  the 
test  program  have  a  fair  chance  for  success, 
qualified  and  motivated  instructors  were 
required.  In  the  imtal  staqes  of  the  proqram, 
experienced  and  motivated  instructors  were  hand 
picked  to  attend  the  Simulator  Instructor  Course 
(SIC).  The  intention  was  to  use  the  same 
instructors  during  tne  entire  period  to 
eliminate  the  variable  of  differing  instructor 
abilities  from  the  transfer  of  training  study. 
There  was  no  problem  retaining  qualified 
instructor  navigators  in  the  test  program,  but 
this  was  not  true  of  instructor  pilots. 

The  instructor  continuity  policy,  two  week 
class  start  interval,  tiqht  summer  manning  and 
instructor  pilot  losses  combined  to  severely 
limit  the  number  of  available  simulator 
qualified  instructors.  Althouqh  the  test  plan 
recommended  the  use  of  the  same  instructors  as 
much  as  possible  to  eliminate  variability  of 
instruction,  this  was  not  feasible  for  the 
pilots.  Considerable  extra  effort  was  required 
to  train  replacements  within  the  minimal  amount 
of  remaining  tine  available  on  the  device.  In 
contest  to  the  pilot  situation,  the  instructor 
navigator  force  remained  relatively  stable. 

Scheduling  and  Profile  Changes:  Another 

variaFte  that  cours’e  3eveTopers  endeavored  to 
hold  constant  was  the  content  of  the  flying 
mission  profiles.  From  the  third  mission 
through  evaluation,  the  profile  contains  a  SKE 
formation  route,  airdrop  and  approach  followed 
by  two  visual  format‘on  low  level  routes, 
airdrops  and  visual  recoveries.  The  test 
program  differed  from  tne  existing  profiles 
beginning  ac  mission  three.  Missions  three  and 
four  were  proposed  to  be  visual  formation 


missions  to  balance  the  heavy  SKE  emphasis  of 
four  simulator  missions.  The  remaining  missions 
and  the  evaluation  were  to  concentrate  on 
SKE/visual  profiles  with  the  intent  being 

alanced  mission  emphasis  prior  to  the 
evaluation  and  course  completion. 

Tne  profiles  actually  flown  during  the  test 
program  did  not  adhere  closely  to  the  guidelines 
for  either  the  normal  or  the  test  program 
profiles.  During  the  test  classes,  it  was 
impossible  to  fly  the  test  group  independently 
of  the  control  group.  Only  infrequently  did  all 
the  test  group  aircrews  make  up  a  formation  for 
which  only  visual  formation  events  were 
planned.  Discussion  of  the  effects  of  the  heavy 
emohasis  on  SKE  by  the  test  group  will  be 
considered  under  the  Conclusions  section. 

The  end  result  was  that  although  course 
developers  had  hoped  to  test  a  specific  sequence 
of  simulator  and  flying  missions,  scheduling 
produced  a  hybrid  sequence  of  missions  Kased  on 
what  existed  arid  what  was  desired.  The 
all-visual  missions  of  the  test  group  were  never 
realized.  Seven  rather  than  six  missions  were 
actually  flown,  as  a  rule. 

Scheduling  and  Proficiency  Advancement:  The 
couries  administered  by  this  traininq  group 
operate  under  the  concept  of  "proficiency 
advancement".  Proficiency  advancement  is  an 
operating  theory  under  which  each  student  must 
demonstrate  proficiency  at  a  task  before  he  or 
she  can  advance  to  the  next  phace  of  training  or 
be  recommended  for  evaluation.  The  lack  of  true 
proficiency  advancement  was  found  to  be  based  on 
a  limited  flexibility  in  the  scheduling  of 
flyinq  time,  constraints  arising  from 
simultaneous  training  in  multiple  crew 
positions,  an  informal  instructor  rating  system, 
and  the  constraints  associated  with 
accomplishing  trainina  events.  These  factors 
tend  to  discouraqe  proficiency  advancement  and 
cause  the  vast  majority  of  students  to  fly  aoout 
the  °ame  amount  of  time  each  class.  This 
problem  is  further  discussed  in  Conclusions. 

RESULTS 

The  results  of  the  test  proqram  will  be 
presented  in  two  parts.  The  first  part  will 
deal  with  the  students  and  the  second  part  will 
consider  the  instructors.  While  there  is  some 
overlap  in  these  two  areas,  for  the  most  part, 
they  are  distinct  topics. 

Students  *  The  test  program  encompassed  30 
student  pilots  and  IS  student  navigators.  The 
data  compiled  on  these  test  subjects  and  the 
control  group  will  be  presented  here. 

The  pilot  students  were  well  qualified  with 
ar  average  of  21  IS  flying  hours  (1631  nrs 
C-130).  The  copilots  were  mostly  recent  UPT 
graduates  with  an  average  of  438  total  hours  (42 
hrs  C-130).  The  navigators  had  a  mix  of 
experience  levels  ranging  from  7000  hours  to  140 
hours  with  a  total  flying  average  of  1924  hours 
(6S0  hrs  C-130). 
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Students  completed  critiques  which  rated 
individual  areas  of  course  desiqn  and 
effectiveness.  A  five  point  scale  was  employed 
with  catagories  labeled  from  "poor"  to 
"outstanding".  Critiques  were  completed  at  the 
end  of  both  the  simulator  and  flying  phases. 
The  ratings  were  consistently  excellent  to 
outstanding  in  most  areas.  Of  particular 
interest  were  the  overall  ratings  (pilot  and 
navigator  responses  combined)  at  the  end  of  the 
flyinq  phase.  Seventy-six  percent  of  the 
students  rated  the  simulator  excellent  or 
outstanding  as  a  transition  to  the  aircraft. 
Sixty-nine  percent  of  the  students  similarly 
rated  the  program  outstanding  as  a  transition  to 
the  flying  phase. 

In  addition  to  the  ratings,  the  students 
made  comments  on  the  critique  forms.  Some  of 
the  comments  dealt  with  suggested  char.qes  in  the 
missions,  such  as  more  or  fewer  malfunctions. 
These  suggestions  were  acted  upon  when 
feasible.  The  size  of  the  list  was  deceptively 
long.  Some  of  the  comments  were  contradictory 
and  thus  heir  validity  is  suspect.  For 
instance,  some  students  in  class  81-016 
recornnended  elimination  of  SKE  lead  time  while 
others  recommended  an  increase.  The  remainder 
of  the  unresolved  comments  will  be  studied 
further  to  improve  the  syllabus.  The  largest 
comment  area  was  praise  for  the  course  as 
benef icial . 

Averaqes  for  the  number  of  sorties  and 
flying  time  expended  for  training  classps  during 
the  sumner  of  1981  were  tabulated.  The  results 
show  that  the  test  qrcups  experienced  fewer 
average  sorties  and  flying  time  than  the  control 
groups,  but  not  by  the  marqin  hypothesized  in 
the  Method  section.  This  information  is 
summarized  under  Program  Averages  (figure  1). 
It  should  be  noted  for  class  81-014  and  Sl-Olo 
that,  although  the  students  completed  training 
with  fewer  flying  hours  and  number  of  sorties, 
the  test  group  required  more  training  days  than 
the  control  group.  The  cause  of  tnis  anomaly 
can  be  traced  to  the  profile  changes,  and  the 
effect  of  the  increased  number  of  training 
events  generated  by  the  use  of  the  AID 
(discussed  under  Conclusions). 

A  complete  listing  of  all  the  control  group 
flight  evaluation  discrepancies  was  compiled  tor 
pilots  and  navigators  respectively.  A 
conparisof  was  made  of  iiscrepamy  areas  and 
frequencies  between  test  and  control  groups  of 
pi l  >t s  and  navigators.  This  data  shows 
Siqmf  icantly  fewer  discrepanc  ies  in  the  test 
qro- p  for  ShE  enroute  formation  position  for 
pilots.  ShE  departure  and  bM.  recovery 
discrepancy  rates  are  appr  imate  1  y  the  same  for 
control  and  test  groups  in  relation  to 
respective  populations.  trend  can  oe  Seen  in 
naviqator  discrepancies  when  comparing  test  md 
control  groups  except  in  the  area  of  8M 
knowleoqe  and  use.  Overall,  Skt  knowledge  and 
formation  position  flying  discrepanc ies  appear 
to  be  reduced  by  the  inclusion  of  US  r.iss  ions 
in  the  syllabus. 


Also  in  figure  1,  the  overall  percentages 
show  that  test  group  pilot  and  navigator 
students  completed  training  without 
discrepancies  more  often  than  those  in  the 
control  group.  The  test  group  accomplished  this 
with  fewer  flying  sorties  arid  hours.  Tnere 
seems  to  be  an  insignificant  difference  in  tne 
number  of  days  in  training  between  test  and 
control  groups  for  the  pilots.  The  navigators, 
in  contract,  show  a  difference  of  about  four 
days.  The  significance  of  all  these  areas  with 
reference  to  the  utility  of  the  ATI)  with  be 
further  discussed  in  Conclusions. 

Instructors:  Based  on  the  Instructor  Mission 
Report  ratings  there  seems  to  be  no  identifiable 
trend  to  the  usage  pattern  of  the  simulator. 
Reliability  rates  for  the  device  will  be 
discussed  later  in  this  section.  The  instructor 
pilots  and  navigators  indicated  their 
perceptions  of  device  operation  and  training 
value  with  a  numerical  rating.  The  data 
generally  reflects  a  "good"  rating  for  device 
operation  and  a  "qood"  to  "excellent"  rating  for 
training  accomplished.  There  is  a  high 
correlation  between  the  device  operation  rrtinq 
and  the  training  accomplished  rating. 

In  addition  to  the  ratings,  the  instructor 
comments  were  compiled  from  the  instructor 
mission  reports.  Also  tabulated  were  the 
frequency  of  the  comment,  area  of  responsibility 
and  the  status.  Numbers  of  comments  declined 
over  the  course  of  the  test  as  the  proqram  was 
"debugged".  A  listing  of  maintenance  related 
comments  extracted  from  all  ot  tne  instructor- 
mission  reports  was  correlated  with  frequency  of 
occurrence  and  numbers  of  ineffective  sorties. 
The  data  shows  that  of  the  6U  simulator  periods 
required  to  support  lb  student  ^.rews  (3  classes 
x  4  crews  x  4  missions  ♦  1  class  x  4  crews  x  .1 
missions),  10  periods  were  lost  and  had  to  be 
rescheduled  for  an  overall  ineffective  rate  of 
17X.  There  seems  to  be  a  decline  in  the  number 
of  maintenance  related  comments  over  the  course 
ot  the  program,  but  the  number  of  ineffective 
sorties  seems  constant.  Tne  predominant 
maintenance  problem  varied  from  class  to  class. 
For  instance,  hydraulic  control  locdinq  was  a 
problem  during  class  81-014  while  motion 
platform  jerkinq  and  software  problems  affected 
classes  8l-01b  :nd  81-018  respect ively.  The 
problems  listed  are  fairly  evenly  distributed 
between  hardware  and  software.  Additional 
training  time  was  lost  or  tne  content  degraded 
by  less  significant  equipment  malt  un^t  ions  that 
went  unrecorded. 

As  with  the  unresolved  student  comments, 
some  instructor  comments  on  the  same  topic  are 
contradictory  and  the.r  validity  is 
questionable.  To  resolve  the»e  t cntradict ions 
and  other  comments,  an  after  action  meeting  was 
held  on  30  Oct  Hi  with  all  available 
instructors.  Tne  remainder  of  the  comments  will 
also  be  studied  further  to  improve  the  syllabus 
and  operat ions /maintenance  interaction. 
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PROGRAM  AVERAGES 


PILOTS 

NAVIGATORS 

Statistical  Area 

Test  Group 

Control  Group 

Test  G;  oup 

Control  G.-oup 

Ranks 

- 2lT 

y 

48 

S 

13 

1LT 

7 

0 

2 

CAPT 

18 

}  7 

1 

4 

MAJ 

4 

16 

4 

1 

LTC 

1 

8 

2 

0 

TOTAL  NO.  OF  SUBJECTS 

30 

no 

15 

20 

SORTIES  PRIOR  TO  RECOMMENDATION 

7.2 

8.8 

7.6 

8.4 

HOURS  PRIOR  TO  RECOMMENDATION 

32.1 

3S.0 

N/A* 

N/A* 

#  OF  DAYS  TO  COMPLETE  FLY  PHASE 

20.7 

20.3 

22.6 

IS. 3 

CHECKS  IDE  RESULTS 

0-1 

2b  -  87% 

79  -  72% 

11  -  73% 

14  -  70% 

Q-l/2 

1  -  03% 

21  -  19% 

0  -  00% 

3  -  15% 

0-2 

3  -  10% 

6  -  05% 

1  -  06% 

1  -  05% 

Q-3 

C  -  00% 

4  -  04% 

3  -  20% 

2  -  10% 

*  Data  not  available.  Not  considered  relevant  due  to  use  of  only  pilot  data  for  flying  program 
schedul inq. 


Figure  1 


CONCLUSIONS 

Primary  Findings 

Transfer  of _ Training:  A  transfer  of 

trailing  ratio  o7  73  8  was  originally 
hypothesized.  Based  on  a  program  of  four 
simulator  missions,  this  rate  would  suqqest  an 
approximate  savinqs  of  two  flight  missions  while 
noldinq  training  standards  constant.  The 
subjective  and  objective  data  collected  by  this 
study,  with  some  qual if icat ion,  support  the 
hypothesis. 

The  overall  flight  evaluation  results  (see 
figure  i)  dearly  show  that  U-i  rates  were  not 
degraded  witn  the  adoption  of  the  ATP.  The 
pilot  data  even  suqqest s  a  slight  improvement  in 
this  ra*e.  In  the  specific  subareas  related  to 
S&E  prccedi-**-  th^re  was  a  significant 
improvement  for  Loth  .ulots  and  navigators.  For 
pilots  there  was  a  891  uecrease  in  the  number  ot 
5*E  related  disc *epanc its.  For  navigators  there 
was  a  1001  decrease  (the  actual  number  of 
discrepancies  declined  from  two  to  zero). 

The  numbe'-  ot  aircraf  t  missions  ?  lown  p**ior  to 
evaluation  (sorties  use  rate)  also  declined  with 
the  addition  of  the  AID.  The  decrease  was  l.b 
and  .8  sorties  for  the  pilots  ano  navigators 
respectively.  Although  this  decrease  do*>  not 
f :  11  y  support  the  hypothesised  transfer  of 
training  rate,  there  is  evidence  that  this  rate 
was  adverse!^  affected  by  factors  unrelated  to 
training.  This  subject  will  a«»  discussed  unde" 
weaknesses  of  the  Study  in  this  section. 


Student  and  instructor  feedback,  as  derived 
from  critiques  an  i  mission  reports,  strongly 
supported  the  use  cf  tne  ATP  for  $KE  training. 
On  30  Oct  81  an  after  action  meet  inq  was  held 
with  ail  available  instructors  who  had 
participated  in  the  SKE  test.  The  consensus 
recommendation  for  future  simulator  use  was  a 
program  ot  four  simulator  missions  and  six  or 
seven  flying  missions  plus  a  flight  evaluation. 

Course Structure:  Data  from  this  study 

suqqests  that  a  block  of  simulator  missions  is 
not  the  most  effective  or  efficient  structure 
for  use  of  the  ATI). 

The  addition  of  simulator  missions  to  tne 
treimnq  program  increases  the  total  number  of 
training  events  in  the  flying  training  phase 
from  fourteen  to  sixteen.  This  increase  in  the 
number  ot  events  caused  an  increase  in  the 
number  of  days  required  by  the  test  group  to 
complete  the  course  (see  figure  I).  In  the 
interests  of  safety,  instructors  are  usually 
restricted  to  a  maximum  of  three  actual  r lying 
missions  per  week.  By  integrating  simulator 
missions  in  the  flying  phase  the  greatest  number 
of  training  events  can  be  accomplished  in  the 
time  al lotted. 

The  integrated  structure  may  also  be  the 
.rest  effective  use  of  the  ATD  from  a  transfer  of 
traminq  point  of  view.  Instructors  noted  a 
weakness  in  the  blocked  schedule  usco  for  the 
t.  st.  The  lest  plan  called  for  a  secuer.ee  of 
two  visual  flight  missions,  four  S*E  simulator 
missions,  then  the  remaining  flying  missions. 
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Instructors  pointed  out  that  students  were 
inclined  to  forget  visual  procedures  du^inq  the 
concentration  on  SKE  in  the  simulator. 
Instructors  felt  an  integrated  approach  would 
make  better  use  of  all  missions.  This  was  the 
recommendation  of  the  instructors  attending  the 
after  action  meeting. 

Courseware :  Courseware  included  a  variety 

of  guides  and  job  aids  desiqned  to  assist  the 
students  and  instructors  in  the  use  of  the  ATD., 
One  of  the  objectives  of  this  study  was  to  prove 
these  support  materials.  To  some  degree  this 
effort  was  hampered  because  much  of  the 
courseware  was  revised  in  response  to  student 
and  instructor  comment  during  the  test.  Thus, 
the  courseware  as  an  independent  test  variable 
was  not  held  constant.  However,  hasea  on 
positive  feedback  from  instructors  and  students, 
plus  the  positive  transfer  of  training  rate  for 
the  program,  the  test  basically  proved  the 
efficiency  and  validity  of  the  materials, 
further  testing  for  validation  is  suggested 
under  Recommendations. 

instructor  Training:  Sufficient  simulator 

instructors  were  qualified  to  complete  the 
study.  There  was  no  specific  data  collected  on 
the  relative  competence  of  these  instructors  but 
it  may  be  assumed  from  the  positive  overall 
study  results  that  minimi,  i  competence  was 
attained.  There  were  two  proqrams  used  for 
instructor  quail f ication.  The  first  was  a 
highly  structured  program  includinq  an  academic 
block  and  a  hands-on  traininq  block.  The 
instructor's  after  action  meetinq  recommended 
specific  improvements  to  this  program.  They 
are:  1.  Reduce  the  length  of  the  academic 
phase,  2.  Increase  the  amount  of  hunus-on 
training,  and  i.  Durinq  hands-on  training, 
include  traininq  missions  with  actual 
students.  A  less  formal  Check  out  program  was 
used  to  make  up  for  instructor  attrition  during 
the  test.  This  program  involved  "piggy  backing” 
instructor  candidates  on  training  missions  with 
fuilv  Qualified  instructors.  Althouqh  mis 
program  is  less  desirable  than  trie  first,  it  ul 
meet  the  need  for  qualified  instructors. 

Additional  Fi tidings 

Maintenance  Support:  Alp  maintenance  had  a 
.major'Tmpict  on  the  test  program.  The  test  was 
hampered  by  hardware  and  software  deficiencies 
throughout  it',  run.  Some  deficiencies  were  the 
revj’t  of  Mcorvct  inivia’  ues'-jn  white  ofnee% 
«ere  due  to  maintenance  manning  and  sol. 
levels . 

Some  deficiencies  ivsim  uncorrected  due  to 
tne  tow  maintenance  manning  ;nd  training  levels 
which  currently  exist.  Manning  levels  for  the 
I  f  S  will  t  itprnyr  as  the  old  Simulators  are 
decoeniissi  sired.  It  is  to  be  hoped  that 
knowledge  levels  in  tne  maintenance  ranks  will 
increase  with  the  Conversion  of  personnel  *ror 
analog  to  digital  systems  and  with  r*sre 
experience  maintaining  this  dxv'.te. 


Training  Capabilities:  Some  important 

traininq  capabilities  were  not  designed  into  the 
AID.  For  instance,  one  important  training  task 
is  the  performance  of  SKE  procedure  turn 
recoveries  in  the  wing  position.  A  capability 
of  th°  simulator  to  train  this  task  was  never 
contracted  for  and  thus  never  designed.  As 
another  example,  trainers  desired  to  use  the 
concept  of  "backward  chaining".  This  concept 
refers  to  a  way  of  training  a  task  which  is  made 
up  of  a  series  of  chained  subtasks.  The  final 
subtask  is  practiced  first,  then  the  last  two 
subtasks,  then  the  last  three,  and  so  on  until 
the  entire  task  is  practiced.  This  concept 
works  particularly  well  when  the  last  subtask  is 
the  most  difficult,  since  the  last  subtask  is 
the  most  practiced.  In  airdrop  training,  the 
final  subtasks  are  the  most  difficult  to  master 
and  thus  this  technique  could  have  proved  very 
useful.  However,  the  oesign  of  the  SKE  computer 
program  required  the  trigqers  of  a  departure, 
climb,  descent  and  slowdown  in  order  to  make  an 
airdrop.  Multiple  approaches  to  the  drop  zone 
cannot  be  accomplished  without  flying  an  entire 
route.  At  some  future  date  this  basic 
programming  may  be  rewritten,  bat  these  training 
events  cannot  be  accomplished  at  this  time. 

In  addition  to  deficiencies  in  the  design 
ana  initial  programming  of  the  device,  some 
other  features  of  the  device  were*  unuseable. 
The  automatic  profiles,  performance  measurement, 
prerecorded  demonstrations  and  auto  message 
features  all  had  a  questionable  reliability 
recurd.  Their  intermittent  operation  caused  a 
degree  o (  frustration  in  the  instructor  tanks. 
A  large  number  of  software  changes  will  be 
required  before  these  features  are  useable. 

The  user/trainer  should  not  expect  perfect 
performance  from  a  prototype  ATD  oaring  the 
installation  and  testing  phase.  Eventually, 
logistic  and  maintenance  support  should  meet 
expectations.  long  procurement  lead  times  are 
to  be  expected  on  software  and  hardware  items 
tor  a  new  device.  The  procurement  contract 
t’auses  that  specified  testinq  in  tne  plant  and 
at  the  site  and  a  data  freeze  date  or  197/  may 
have  provided  some  protect  ion  from  defect  in  the 
AID,  but  they  also  extended  the  time  at  which 
;f)dintena»it  _•  and  logistic  support  will  catch  up 
with  desired  training  quality. 

'we  aw  nesses  of  the  Study:  It*  the  method 

section,  the  course  developers  proposed  to 
extract  result,  from  .he  grade  sheets  to  Support 
collection  of  objective  data.  Inis  was  not  done 
because  of  the  limited  value  of  this  data, 
whenever  ar.  evaluator  remembered  to  complete  the 
evaluation  column  of  this  for-?,  a’  areas 
applicable  to  Sk:  forma* ton  position  and 
procedures  were  usually  graded  a!  trie  minimum 
level  of  proficiency.  The  few  evaluators  who 
avoided  this  central  tendency  and  \hO»ed  some 
variation  its  performance  and  knowledge  levels  do 
not  represent  a  numerically  significant  group 
for  study. 

?r»t\  study  has  limited  value  because  of  the 
manner  in  which  the  objective  data  was 
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collected.  In  the  pilot  mission  qualification 
course,  there  ,>re  no  specified  criteria  for  the 
required  level  of  proficiency  in  flying  the  SKE 
wing  position.  Tnis  position  is  flown  4,000 
feet  in  trail  for  the  number  l  winqman  and  8,000 
feet  in  tra4l  for  number  3.  A  criteria  such  a* 
"maintain  4,000  feet  in  trail  as  number  2 
winqman  ♦  '  000  feec"  does  not  exist.  There  are 
no  specified  Mmits  in  MACS  60-1,  Aircrew 
Standardization  .  Evaluation  Program,  in  relation 
to  acceptable  1  units  ~o(  Format  ion  position.  The 
SKE  subareas  on  the  evaluator  work  sheet  are 
qr.  led  satisfactory  or  unsatisfactory.  For  this 
study,  course  developers  have  been  forced  to 
rely  on  subjective  evaluator  judgements  of 
formation  position  and  use  the  checkride 
pass/fail  rates  as  objective  data. 

The  control  of  var  iablt-s  was  a  major 
weakness  of  this  study.  Too  many  conditions  in 
the  traininq  proq^am  were  allowed  to  chanqe  over 
tne  course  of  the  test.  Traininq  profiles, 
numbers  of  sorties,  instructor  personnel  ana 
other  proposed  parameters  discussed  under 
Problems  and  Results  varied  significantly.  The 
test  program  missions  in  the  new  simulator  were 
developed  to  complement  the  existing  flyinq 
proqrar’\  If  the  simulator  had  been  an 
established  training  device,  a  chanqe  in 
traininq  policy  would  required  validation 
of  a  modified  flyinq  program.  Neither  of  these 
approaches  is  optimal.  A  training  syllabus  that 
teaches  required  tasks  should  be  prepared  and 
then  traininq  time  apportioned  to  thv.  AID'S  or 
flyinq  traininq  based  on  the  most  effective  and 
efficient  utilization  of  these  resources. 
Exercise  of  control  over  all  phases  of  the 
training  program  desiqn  would  have  insured  more 
accurate  te«t  results. 

Two  additional  weak  areas  deserve 
discussion:  proficiency  advancement  and  the 
small  number  of  test  subjects.  As  discussed 
under  Procedures,  advancement  was  adversely- 
affected  by  current  scheduling  practices.  As 
discussed  in  this  section,  proficiency  <s  rather 
ill-defined  anu  event  oriented.  When  the 
student  has  flown  all  the  required  events  listed 
on  the  grade  sheet  on  the  required  number  of 
flyinq  missions  established  by  the  Course 
bunnary  l  document,  he  is  qer.erally  considered 
proficient.  In  examining  the  term  "proficiency 
advancement",  as  it  was  applied  to  the  test 
program,  it  is  evident  that  "prof ic iency"  was  a 
subjec‘;»e  evaluation  wit?  little  basis  in 
objective  fact  and  that  "advancement"  was 
inflexibly  based  on  their  sl.^nt's  flying 
schedule.  Neither  of  these  problems  could  be 
overc  )me  in  the  test  program  methodoloqy  by  the 
relatively  small  number  of  test  subjects.  hee 
Recofwnendationx  for  applicability  to  future 
sylMb;  *td  any  further  investigations. 


traininq.  This  allows  instruction  given  in  the 
simulator  to  be  more  compatible  with  that  given 
in  the  aircraft.  This  should  reduce  any 
possible  neqative  transfer  that  could  occur  as  a 
result  of  instructor  idiosyncrasies  (8). 


Recommendat ions 


Pr 4  a ry _ Recommendations :  Until  a 

significant  amount  of  further  data  can  be 
compiled  from  students  in  a  mission  traininq 
curriculum  incorporating  the  IFS,  the  following 
recommendations  are  made  regarding  that 
curriculum: 

11)  The  IFS  provides  good  initial 
Staf ionkeeping  Equipment  training  and  should  be 
integrated  throughout  the  flying  pnase  of 
instruction. 

(?)  The  course  of  instruction  for 
pilots  and  navigators  following  academics  should 
consist  of  four  simulator  missions  interspersed 
with  six  flight  missions  and  an  evaluation. 

(3)  'The  simulator  instructor  candidates 
should  receive  one  day  of  academic  traininq,  two 
simulator  missions  without  students,  three 
training  missions  with  students  end  an 
evaluation  (if  required).  Instructor  training 
should  be  accomplished  using  the  traininq 
syllabus  for  the  instructor's  course  (9). 

(4)  The  courseware  that  was  developed 
for  this  test  program  should  be  formalized  ana 
used  until  validation  on  a  statistically 
significant  student  population  is  completed. 

( b )  Greater  emphasis  on  true 
proficiency  advancement  should  be  supported  by 
managers  and  supervisors.  Training  should  oe 
less  event  oriented  ana  scheduling  handled  with 
more  flexibility. 

Additional  Recommendations:  The  following 
recommendations  are  of  less  immediate 
importance ,  but  should  also  be  implemented: 

(1)  Specific  performance  criteria 
should  be  establisned  .or  tasks  trained  in 
Simulator  ana  flying  training  tor  the  purpose  of 
testinq  and  validation.  These  criteria  will 
promote  standardized  evaluation  uf  student 
performance  by  instructors  and  evaluators . 

(2  continuing  studies  %.-,uuld 
investigate  the  rate  of  proficiency  attainment 
in  simulator  ana  flying  training  to  identify  the 
best  media  tor  instruction. 

(3)  Adequate  AID  time  should  be 
allocated  for  course  development  efforts. 


In  factor  Ut  i  1  Izat  ion:  A  final  point  to 
i  ‘.sure  continued  traTningT et~ fee t i veness  of  tnis 
AID  is  the  single  instructor  concept.  Even 
though  this  variable  has  not  been  adequately 


studied,  there  appears 
effectiveness  when  a 
rexoonsible  for  both 


to  be  an  increase  in 
Single  instructor  is 
simulator  and  flying 


(4)  Continuing  efforts  should  be  made 
to  improve  ATD  maintenance  support  and  ATD 
reliability. 


(S)  A  concerted  effort  should  be  made 
to  improve  if  3  software  so  that  all  design 
capabilities  of  th**  device  *re  fully  useable. 
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Refining  these  features  will  ease  instructor 
workloads . 

(6)  Every  effort  should  be  inade  to 
increase  supervisory  awareness  and  support  for 
test  programs  and  validation  studies. 

(7)  Standardization  and  Evaluation 
should  recognize  the  efftcti veness  of  this 
device  for  student  training  as  described  in  this 
report  or  conduct  their  own  validity  assessment. 

It  is  through  periodic  management  reviews 
and  studies  of  this  type  that  training  policies 
are  examined  and  constructive  changes  made  to 
improve  training  techniques. 
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1  ABSTRACT 

The  Instructor/Operator  Station  (IOS)  of  a  simulator  is  the  focus  of  user  control  of 
training  exercises.  Design  of  the  IOS  is,  therefore,  crucial  if  the  user  is  to  carry  out 
his  training  roles  and  responsibilities  efficiently.  Unfortunately  IOS  designs  have  often 
been  pocjrly  human  engineered  and  have  been  based  on  insufficient  training  and  human  factors 
data.  This  paper  (1)  assesses  IOS  designs  for  current-generation  Operational  Flight 
Trainers  (OFT)  and  Weapon  Systems  Trainers  (WST),  (2)  identifies  strengths  and  weaknesses, 
(3)  develops  an  approach  to  improved  IOS  design,  and  (4)  develops  alternative  IOS  configur¬ 
ations.  The  alternatives  are  comparable  in  training  capability.  They  differ  In  their 
primary  methods  of  control:  fixed-function  console-mounted  electronic  touch  pads  versus 
va r ia b 1 e— f unc t ion  CRT  touch  panels.  The  primary  conclusion  of  the  paper  focuses  on  the 
importance  of  systematic  application  of  training  data  with  the  resuUant  potential  for 
improvement  in  IOS  design  and  operations^. 


been  incorporated  into  the  IOS.  All  too  often 
selection  and  implementation  of  the  capabili¬ 
ties  have  been  based  on  insufficient  analyses 
and  data.  As  a  result  IOSs  and  the  associated 
software  have  been  poorly  designed. 

Training  devices  have  been  delivered  with 
IOSs  which  are  complex  and  diffic:  It  to  oper¬ 
ate.  This  has  led  to  user  frustration  in  not 
being  able  to  use  the  device  to  its  training 
potential.  For  example,  many  IOSs  have  multi¬ 
ple  data  entry  methods  such  as  switches,  light 
pens,  numeric  keyboards,  alphanumeric  key¬ 
boards;  multi-function  switches;  pages  and 
pages  of  CRT  information  which  must  be  called 
up  before  the  specific  item  can  be  located, 
etc.  Extensive  formal  training  Is  required 
for  the  instructor/operator  to  become  profi¬ 
cient.  Daily  use  is  required  to  remain  famil¬ 
iar  with  the  operation  of  the  IOS.  In  the 
real  world"  situation  the  insti uctor/operator 
may  or  may  not  have  had  formal  training.  More 
likely  he  has  received  an  abbreviated  on-the- 
job  training  program  and  does  not  use  the 
device  daily.  Many  Instructors  are,  there¬ 
fore,  not  qualified  to  administer  training  In 
the  device.  The  result  is  that  training  qual¬ 
ity,  quantity,  standardization,  and  efficiency 
suffer. 


INTRODUCTION  ^ 

In  the  past  decade  the  states-of-the-art 
in  both  training  systems  and  simulator  tech¬ 
nology  have  advanced  rapidly.  Innovation  in 
training  systems  has  resulted  from  the  use  of 
the  procedures  and  techniques  inherent  in 
Instructional  Systems  Development  (ISD). 
Simulator  hardware  and  software  advances  have 
been  numerous  and  varied  (e.g.,  computer  and 
software  capabilities,  visual  system  tech¬ 
nology,  data  entry  methods  and  da^n  manipula¬ 
tion  techniques).  These  innovations  and  a 
variety  of  other  factors  (e.g.,  fuel  costs, 
safety,  etc.)  have  led  to  escalation  of  the 
use  of  training  devices.  Indications  are  that 
this  trend  will  continue  in  the  foreseeable 


In  many  instances  it  appears  that  tech¬ 
nology  has  been  the  driving  force  in  training 
device  design.  Devices  have  been  designed 
because  the  technology  existed,  not  because  an 
analysis  based  on  training  requirements  deter¬ 
mined  the  best  candidate  and  its  capabilities 
to  meet  the  training  requirement.  The  results 
have  been  that  money  has  been  spent  for  tech¬ 
nology  with  little  regard  to  training  effec- 
t i veness . 


The  Instructor/Operator  Station  (IOS)  has 
been  the  recipient  of  much  of  the  explosion  of 
simulation  technology.  As  technology  has 
permitted  higher  fidelity  simulation  of  more 
and  more  trainee  tasks,  it  has  also,  enabled 
expanded  capabilities  to  control  the  simula¬ 
tion  environment .  These  capabilities  have 


IOS  design  is  not  just  the  physical  layout 
of  n  work  place.  It  also  encompasses  consid¬ 
eration  of  Instructional  features,  information 
management,  software  capabilities,  and  many 
other  varied  factors.  To  properly  incorporate 
the  many  factors  into  an  efficient  training- 
effective  design  requires  a  systematic 
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analysis  process.  It  is  apparent  that  many 
lOSs  currently  in  use  have  not  evolved  through 
such  an  analysis  and  design  process. 

PURPOSE 

The  purpose  of  this  paper  is  to  present 
the  results  of  a  systematic  analysis  of  past 
and  current  IOS  designs,  assess  rheir 
strengths  and  weaknesses,  develop  design  prin¬ 
ciples  to  enhance  the  strengths  and  to  elim¬ 
inate  the  weaknesses,  and  implement  the  prin¬ 
ciples  in  alternative  generic  IOS  designs. 
The  orientation  of  the  paper  is  toward  the 
identification  of  problems  and  the  presenta¬ 
tion  of  proposed  solutions.  The  proposed 
solutions,  if  implemented,  will  provide  an  IOS 
which  is  readily  usable  and  more  training 
effective.  In  addition,  reliability  and  main¬ 
tainability  will  he  improved. 

Emphasis  is  on  the  IOS  of  operational 
flight  trainers  (OFT)  and  weapon  system  train¬ 
ers  (WST)  for  fighter/attack  aircraft.  Proce¬ 
dures,  results,  and  recommendations,  however, 
are  also  relevant  to  many  other  simulator 
appl i cat  ions. 

IOS  WEAKNESSES 

There  are  significant  differences  in  the 
design  features  of  existing  simulator  lOSs. 
Some  work  hotter  than  others;  some  have 
sophisticated  features,  hut  do  not  work  very 
well;  others  are  simple  and  work  quite  well. 
Collectively,  however,  there  .ire  weaknesses 
which  characterize  OFT  and  WST  lOSs.  These 
weaknesses  are  as  follows: 

•  Layout,  labelling,  coding,  etc.,  do  not 
optimize  device  operation  and  minimize 
instructor  workload. 

•  Steps  required  r.o  access  many  CRT  displays 
are  time-consuming  and  inelticiint. 

•  Delays  in  display  access  cause  inefficien¬ 
cies  in  training  proilem  control  and  moni¬ 
toring  . 

•  D..t  a  entry  methods  are  contusing,  redun¬ 
dant  ,  and  i ne ft i c i ent . 

•  Device  complexities  make  it  difficult  for 
instructors  to  Implement  changes  or  mod i t v 
missions  to  meet  student  needs. 

•  Many  features  and  capabilities  are  not 
needed,  not  used,  or  difficult  to  use. 

•  Simulator  oupabi 1 i t les  are  not  keyed  to 
tin  training  requ'ronents  (i.e.,  objec¬ 
tives)  of  the  training  system  in  which  it 
will  be  used.  The  simulators  art-,  there¬ 
fore,  not  properly  integrated  into  the 
training  system  as  an  efficient  element 
which  fills  a  cleariy-det ined  need. 


There  are  other  weaknesses  which  do  not 
involve  IOS  design  features,  but  which  signif¬ 
icantly  affect  IOS  operation  and  use.  These 
weaknesses  are  as  follows: 

•  Instructor  training  does  not  adequately 

prepare  instructors  for  their  roles  and 
responsibilities  in  using  simulators. 

•  Instructor  roles  and  responsibilities  are 
not  completely  defined  and  implemented. 

•  Student  training  syllabi  are  poorly  devel¬ 
oped  and  used.  Standard! zat ion ,  organiza¬ 
tion,  efficiency,  and  quality  of  instruc¬ 
tion  are,  therefore,  impaired. 

•  Instructor  handbooks  are  not  properly 

organized  and  formatted  to  provide 

instructor  assistance  in  operating  the 

device  in  a  training  situation.  They  are 

massive  informational  volumes,  not  train¬ 
ing  tools. 

When  taken  together  the  clear  impression 

is  conveyed  that  the  total  concept  of  design 
for  the  instructor  is  very  ‘'loose"  (i.e.,  the 
systematic  principles  of  man-machine  inter¬ 
face,  human  factors,  and  training  technology 
have  not  been  properly  applied).  The  net 
result  is  that  planning  and  design  for  opera¬ 
tion  of  the  device  are  secondary  components  of 
the  total  simulator  development  process. 
Actual  operation  of  the  device  and  training 
using  the  device,  therefore,  suffer. 

DESIGN  PRINCIPLES 

Clear  delineation  of  IOS  weaknesses  pro¬ 
vides  the  basis  for  development  of  problem 
solutions.  The  first  step  in  developing 

specific  solutions  is  to  formulate  a  set  of 
design  principles  from  the  IOS  weaknesses. 
These  principles  translate  the  weaknesses  into 
guidelines  for  improvements.  The  design  prin¬ 
ciples  in  turn  provide  a  framework  for  Jevel 
oping  the  design  concept.  Design  principles 
which  were  derived  from  the  IOS  weaknesses  are 
as  follows: 

•  Reduce  the  i  n«t  rue  t  or '..per  a  tor  require¬ 

ments  for  lormal  '.raining  by  providing,  an 
instructor/oper.itor  instructional  aid 
system  at  or  near  the  IOS.  This  feature 
wi  1 1  not  ally  provide  training  in  the 
operation  of  the  device  hut  will  a*-*  i-ely 
assist  th*'  instructor  pilot  (IP)  il-euyh- 
out  the  training  exercise. 

•  Emphasize  the  use  of  automated  training  as 
the  normal  mode  of  training. 

•  Redi.ce  t'n  i  ns  t  rue  tor /opera  t  or  workload  bv 
automating  the  ancillary  ’asks. 

•  Reduce  the  number  and  types  of  data  entry 
methods  it  the  IOS. 
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•  Use  “touch  panels"  on  CRTs  and  electronic 
"touch  pads"  on  the  IOS  console  as  the 
primary  data  entry  methods. 

•  Design  IOS  layouts  to  enhance  operation, 
interpretation,  sequences  of  actions,  etc. 

•  Eliminate  the  use  of  multi-function  con- 
t  ro  1  s/swi  t  ch.es . 

•  Standardize  the  nomenclature  used  on  con¬ 
trols  to  reflect  IF  terminology. 

•  Investigate  the  use  of  color  on  CRT  dis¬ 
plays  to  highlight  important  points. 

•  Upgrade  student  and  instructor  training 
a  nd  o pe  r a  t i ng  ma t  e  r i a  1 s . 

•  Improve  reliability  and  maintainability  by 
use  of  touch  controls,  and  eliminate  light 
pens,  alphanumeric  keyboards,  etc. 

•  Reduce  cost  by  limiting  IOS  capabilities 
to  defined  training  requirements. 

DESIGN  CONCEPT 

The  design  concept  contains  the  essential 
elements  to  implement  the  design  principles 
and  to  remedy  the  IOS  weaknesses.  It  consists 
of  two  components:  hardware  and  software. 

The  components  interact  and  are  dependent  one 
upon  the  other.  A  major  issue  in  anv  design 
involving  both  hardware  and  software  is  estab¬ 
lishing  the  proper  functional  balance  between 
the  two.  In  pointing  out  the  distinction 
between  hat  'ware  and  software,  there  is  an 
emphasis  that  IOS  design  is  mum  more  than 
using  good  human  factors  principles  in  deter¬ 
mining  what  the  IOS  should  look  like.  It  is  a 
systematic  process  of  determining  the  informa¬ 
tion  storage,  retrieval,  display,  and  manipu¬ 
lation  requirements,  and  of  implementing  these 
requirements  in  a  combination  ol  hardware  and 
software  which  optimizes  instructor  perform¬ 
ance.  Major  design  concepts  are  discussed  in 
the  following  paragraphs. 

Hardware 

The  IOS  hardware  must  be  reliable,  main¬ 
tainable  and  eas_.  to  operate,  and  yet  must 
contain  the  components  required  for  control, 
monitoring,  and  evaluation.  Major  hardware 
components  of  tne  proposed  IOS  design  concept 
are  as  follows: 

•  The  main  conso'e.  The  main  console  is 
comp;ict ,  and  is  designed  lor  a  single  instruc¬ 
tor/operator .  All  layout  is  consistent  with 
good  human  factors  design  principles.  Partic¬ 
ular  emphasis  is  on  ease  of  "sc  of  controls, 
orientation  of  CRTs  for  ease  of  display  inter¬ 
pretation,  and  a  functional  work  surface  which 
accommodates  instructor  guides  ind  checklists. 

•  Instructor  Aid  System  (IAS)  The  IAS  Is  a 
separate  small  CRT  with  associated  controls. 
It  is  an  integral  part  of  the  IOS  and  serves 


two  functions:  (1)  presents  instruction  on 

basic  trainer  operation  and  (2)  prompts  and 
guides  instructors  during  the  course  of  train¬ 
ing  exercises.  It  may  also  bo  used  to  display 
selected  cockpit  instruments  (e.g.,  multi- 
lunction  display,  horizontal  situation  indi¬ 
cator,  etc.). 

•  Instructor's  control  panel.  The  instruc¬ 
tor's  control  panel  is  the  major  "hard-wired" 
part  of  the  IOS  dedicated  to  simulator  con¬ 
trol.  It  consists  primarily  of  fixed-function 
electronic  touch  pads. 

•  Monitors^.  The  major  components  for  data 
manipulation  and  display,  trainee  evaluation, 
and  situation  displays  are  CRT  displays. 
Numbers,  sizes,  and  locations  of  CRT  displays 
are  dependent  upon  the  specific  application. 
A  typical  application,  as  presented  later  in 
this  paper,  consists  of  three  displays:  one 
to  monitor  the  visual  scene  and  two  to  display 
data . 

Sof  tware 

In  order  to  combine  simplicity  of  hardware 
with  complexity  of  weapons  systems  and  train¬ 
ing  problems,  it  is  essential  that  software  be 
designed  to  enhance  instruct'  r  performance. 
The  software  must  he  simple  to  manipulate, 
present  information  in  easily  used  formats, 
and  facilitate  problem  control  and  monitor¬ 
ing.  Major  features  of  the  proposed  software 
design  concent  to  meet  these  goals  are  as 
foil ows : 

•  Display  continuity.  A  given  display  is 
always  presented  on  the  same  CRT.  There  is  no 
switching  of  displays  between  CRTs  at  tin' 
instructor’s  discretion.  When  a  display  is 
Selected  or  is  called  up  aut omat i ca 1 1 y  by  the 
software,  it  always  appears  in  the  same  place. 

•  Standardized  displays.  Each  type  of 
display  has  a  distinct,  precisely  prescribed 
format  wliica  is  always  used  for  that  type  of 
display.  Formats  are  developed  to  enhance 
interpretation  and  use  of  the  information 
presented.  Display  highlights  are  emphasized 
by  spacing,  graphics,  bold  a  1  phantimer  i  c  s  , 
etc.  It  is  also  recommended  that  color  be 
used  to  highlight  portions  of  tF<  disrluvs. 

•  Automatic  presentation  of  displays.  Maxi¬ 
mum  use  is  made  of  software  selection  of  dis¬ 
plays,  so  that  minimum  instructor  intervention 
is  required.  During  pre-programmed  modes  all 
displays  are  software  selected.  During  free 
flight  the  amount  of  instructor  display  selec¬ 
tion  Is  minimized  through  keying  displays  to 
trainee  tasks. 

•  Minimize  steps  to  change  displays .  A  s 
discussed  -ibovt*,  display  location  is  stand¬ 
ard.  There  is,  therefore,  no  concern  with 
where  a  gwen  display  will  appear.  Also 
displays  which  are  candidates  to  replace  a 
given  display  being  presented  on  a  CRT  are 
selectable  using  the  touch  pads  on  the  CRT 
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face.  In  most  cases  changing  displays  will 
require  a  one-touch  step.  In  most  other  cases 
software  design  should  minimize  the  number  of 
decision  points  (i.e.,  steps)  the  instructor 
must  handle. 

•  Minimum  steps  to  manipulate  training  exer¬ 

cises.  Minimum  steps  to  manipulate  is  closely 
reiated  to  minimum  steps  to  change  displays. 
In  this  case,  however,  rather  than  changing 
displays  for  monitoring  purposes  only,  the 
instructor  is  locating  the  display  from  which 
he  will  affect  training  problem  control  (e.g., 
locate  and  select  a  specific  emergency,  locate 
and  activate  a  specific  navigational  beacon, 

etc . ) . 

•  Large  selection  of  programmed  exercises. 

One  of  the  strengths  of  the  proposed  approach 
to  10S  design  and  operation  is  a  high-quality 
front-end  analysis.  Among  other  things,  the 
analysis  yields  a  realistic  training  device 
syllabus  which  is  keyed  to  the  training 
requi foment s  and  training  situation.  The 
programmed  exercises  designed  from  the  sylla¬ 
bus  are,  therefore,  essential  parts  of  the 

total  training  system  and  should  be  adminis¬ 
tered  to  each  student.  The  large  selection 
allows  consideration  of  student  skills  and 
progress,  a  variety  of  equal iy  difficult  exer¬ 
cises  for  different  students,  standardized 
training,  programmed  exercise  capability  for 
all  training  phases,  and  elimination  of  time- 

consuming  set-up  required  in  conventional  free 
flight  exercises. 

•  Large  selection  of  initial  conditions.  On 

many  simulators,  setting  up  initial  conditions 
in  the  free  flight  mode  is  time-consuming,  and 
may  be  nonstandard .  A  large  set  of  initial 

conditions  gives  instructors  the  flexibility 
they  desire  and  improves  use  of  time  and 
standardization.  In  the  proposed  IOS,  selcc- 
1  ion  of  a  given  set  of  initial  conditions 
would  key  other  responses  from  the  !0S.  For 
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Two  generic  alternative  iOS  con;  igur.it  ions 
are  presented.  Both  use  fix***'  function  elec¬ 
tronic  touch  controls  or.  the  IOS  panels  and 
fixed  and  variable  function  touch  panels  on 


the  CRTs  as  the  methods  of  simulator  control. 
The  primary  difference  between  the  two  alter¬ 
natives  is  the  division  between  the  methods  of 
simulator  control.  In  Alternative  1,  the  use 
of  electronic  touch  controls  is  confined  to 
basic  simulator  operation  (e.g.,  power,  motion 
system  operation,  freeze).  Touch  panels  on 
the  CRT  faces  are  the  primary  method  for  con¬ 
trolling  training  exercises. 

In  Alternative  2  there  is  an  increased  use 
of  fixed  function  electronic  touch  controls 
mounted  on  the  IOS  vertical  panel.  These  con¬ 
trols  are  used  to  carry  out  many  of  the  func¬ 
tions  for  which  CRT  touch  panels  are  used  in 
Alternative  1.  CRT  touch  panels  are  also  used 
in  Alternative  2. 

Alternative  1  is  an  extension  of  the 
state-of-the-art  being  used  to  design  the 
current  generation  of  simulators.  Kmphases 
are  on  a  minimum  use  of  hard-wired  controls 
and  a  maximum  use  of  software  control  through 
input  sources  which  are  displayed  as  re¬ 
quired.  Alternative  2  is  an  update  of  the 
basic  approach  used  in  earlier  generation  sim¬ 
ulators  (i.e.,  extensive  use  of  hard-wired 
controls)*  State-of-the-art  technology  is 
used  in  both  the  control  methods  chosen  for 
Alternative  2  (i.e.,  electronic  touch  pads  and 
CRT  touch  panels)  and  problem  control  software. 

These  two  alternatives  are  the  two  ends  of 
a  continuum,  one  extreme  of  which  is  CRT  touch 
panels  and  the  other  extreme  of  which,  is  con¬ 
sole-mounted  electronic  touch  pads.  Although 
Alternative  2  does  not  involve  complete  con¬ 
trol  via  electronic  touch  controls,  it  is  con¬ 
sidered  to  contain  the  most  extensive  f oa s i b 1 e 
use  oi  electronic  touch  pads  in  light  of  the 
current  technological  state-oi -the-art .  More 
extensive  use  of  electronic  touch  pads  in  lieu 
of  CRT  touch  panels  would  lead  to  an  IOS 
design  which  vi.  i  ites  many  of  the  principles’ 
of  compactness,  ease  of  use,  and  information 
management . 

Ollier  candidate  alternatives  in  addition 
to  those  presented  would  fall  between  the  two 
extremis  (i.e.,  would  consist  of  different 
mixtures  of  consol  e-mount ed  elect r  lie  touch 
pads  and  CRT  touch,  panels).  Thcs,  alterna¬ 
tives  wouid  ■'•institute  a  set  of  virtually 
unlimited  permit  a  t  i  ons  of  combinations  of 
electronit  tomb  pads  and  CRT  touch  panels. 
Presentation  of  all  such,  alternatives  would  be 
vlrtuillv  impossible. 

By  providing  the  anchor  points  ot  the  con¬ 
tinuum  (I.e.,  I  tie  t  wo  feasible  extremes) 
rather  than  numerous  alternat ives,  the  con¬ 
cepts  of  operation  oi  each  basic  fOS  l  vp«-  can 
re  clearly  presented  along  with  (tie  c.dvunt  ages 
and  disadvantages  of  each.  Variations  ot  an 
IOS  type  tor  a  specific  application  can  be 
identified  based  on  specific  simulator  re¬ 
quirements  and  personal  preference?.  of  'tub  feet 
Mi t ter  1  Xpert s  (SMFs),  analysts,  and  engineers. 
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Alternative  1 

The  first  alternative  features  a  high 
level  of  automation  to  assist  instructors,  and 
simulator  control  via  CRT  displays  which  con- 
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Al 1  other  controls  are  programmed  to  use  touch 
panels  on  the  faces  of  the  two  CRTs.  Specific 
touch  panels  for  each  display  .ire  programmed 
and  are  presented  in  the  lower  portion  of  the 
d i splay . 

The  selection  of  touch  panels  by  display 
is  based  on  an  analysis  of  the  options  which 
instructors  may  choose  to  exercise  at  i  given 
point  in  a  training  exercise.  The  options  are 
derived  from  the  trainee  and  instructor  tasks 
in  progress  and  the  associated  displays  used 
to  monitor  trainee  performance.  This  philos¬ 
ophy  enables  instructors  to  select  alternative 
displays,  insert  malfunctions,  modify  weather, 
etc.,  without  having  to  carry  out  a  multi-step 
selection  process  through  a  series  of  menus. 
Rather,  selection  alternatives  have  been  built 
into  the  software  md  are  readily  available 
through  the  touch  panels  on  the  two  CRTs. 
This  teature  will  have  a  major  positive  effect 
on  instructor  workload  and  ease  of  10S  ope.'.i- 
t  ion. 

Although  emphasis  is  on  the  use  of  pre¬ 
selected  alternative  actions  during  the  course 
of  a  training  exercise,  the  IOS  has  the  cap¬ 
ability  to  allow  more  flexibility  of  instruc¬ 
tor  action.  Through  the  use  of  a  series  of 
menus  which  tie  back  to  a  master  menu, 
instructors  ar**  able  to  have  greater  control 
over  simulator  operation.  It  is  essential  to 
eaph.asi.ee,  however,  that  through  proper  pro¬ 
gramming  based  on  an  accurate  analysis,  the 
normal  Method  of  operation  will  be  through  the 
programmed  alternatives  presented  on  th*  CRT 
touch  panels.  The  more  time-consuming,  and 
potentially  confusing  processes  of  working 
through  the  selection  menus  Is  meant  to  he  the 
exception  rather  than  the  rule. 

Another  feature  which  reduces  instructor 
workload  Js  extensive  use  of  programmed 


exercises  and  a  large  set  of  initial  condi¬ 
tions.  The  initial  conditions  are  easily 
selectable.  Through  the  software  they  affect 
the  displays  and  alternatives  for  trainee 
tasks  carried  out  in  the  initial  condition 
environment . 

It  is  recommended  that  the  "normal"  mode 
of  operation  be  with  programmed  exercises. 
The  exercises  should  be  derived  from  a  well- 
integrated,  approved  syllabi  s  in  which  the 
simulator  plays  an  integral  role  in  the  train¬ 
ing  sequence.  The  programmed  exercises  facil¬ 
itate  standardization  of  instruction,  hut 
retain  the  capability  for  on-line  modification 
in  -esponsc  to  trainee  performance  (e.g., 
insertion  of  additional  malfunctions,  changing 
environmental  conditions). 

A  large  sc i  of  exercises  (i.e.,  2  5  or 

more)  should  be  programmed  aud  updated  as 
required  to  incorporate  procedural  changes, 
tactics  changes,  etc.  This  will  give  instruc¬ 
tors  the  capability  to  vary  the  scenarios 
within  a  type  and  difficulty  level,  and  also 
to  have  a  wide  range  of  types  and  difficulty 
levels,  available. 

The  free  flight  mode  requires  the  greatest 
instructor  Involvement.  Operations  in  the 
free  flight  mode,  however,  are  designed  to 
reduce  instructor  workload.  This  is  accom¬ 
plished  through  the  IOS  software.  Routine 
decisions  and  selections  are  made  by  the  IOS. 
In  many  cases  instructor  intervention  is  by 
exception.  For  example,  given  that  the 
trainee  is  pe” forming  a  certain  task  which  has 
been  selected  by  the  instructor,  displays  to 
monitor  trainee  performance  will  be  selected 
and  presented  by  the  software.  Also,  the  most 
likely  alternate  selections  and  most  likely 
subsequent  trainee  tasks  will  be  'vailabl.*  as 
CRT  touch  panels.  Cnlj  if  «J  liferent  alter¬ 
natives  are  desired  will  Instructor  inter¬ 
vention  be  required. 

A  large  set  of  programmed  initial  condi¬ 
tions  (l.e.,  40  ur  more)  plays  a  major  role  in 
operation  ol  the  tree  flight  mode.  This  set 
she  old  be  derived  t  rom  a  systematic  ati.ilyi.ls 
of  (1)  conditions  under  which  a  training  exer¬ 
cise  will  begin  and  (2)  conditions  to  which 
the  simulator  will  he  reset  during  the  course 
ot  exercises.  Thiougn  use  of  programmed  ini 
t ini  conditions,  initial  set  up  and  reset 
times  will  be  minimized. 

The  intent  is  to  use  the  initial  eomM- 
t ions  as  programmed,  without  modification.  If 
the  set  of  initial  conditions  is  based  upon  an 
accurate  analysis,  th.  is  should  be  the  case. 
The  software,  however,  contains  tne  capability 
for  on-line  changes  to  sriecud  parameters. 
It  is  recommended  that  vvather  and  stores  be 
changeable  on-line.  Changes  to  any  other 
parameters  would  be  made  off-line.  The 
ability  to  make  changes,  both  to  initial  con- 
dlticr*  a. id  programmed  exercises,  should  be 
tightly  contr.Iied.  Such  changes  should 
result  only  from  periodii  training  conferences 
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Figure  1.  Generic  Instructor/Operator  Station  -  Alternative  1. 


and  reflect  the  consensus  of  the  training 
community.  Otherwise,  the  programmed  events 
may  be  based  upon  the  whims  of  the  individuals 
who  like  to  make  changes. 

Al  tentative  2 

The  second  alternative  also  features  a 
high  level  of  automation  to  aasist  instruc¬ 
tors.  The  major  difference  is  that  many  of 
the  'nstructor  actions  which  employed  CRT 
touch  panels  in  Alternative  1,  employ  fixed- 
function  electron.c  touch  pads  on  the  I  OS  con¬ 
sole  in  Alternative  2.  The  basic  configura¬ 
tion  of  the  Alternative  2  IDS  Is  shown  in 
Figure  2 .  Hard-wiie  controls  are  used  for  the 
fol 'owing  functions.  Noke  that  the  first  six 
(marked  with  an  asterisk)  are  the  same  a-  in 
Alternative  1.  The  remainder  are  additions  to 
Alternative  1,  and  indicate  the  major  hardwa.e 
difference  between  the  two  alternatives. 

•  Operation  of  displays  fe.g.,  power, 

brightness,  contrast).* 

•  Basic  trainer  control  (e.g.,  power, 

freeze,  motion).* 

•  Communications  with  the  trainee  and  main¬ 
tenance.  * 


Miscellaneous  problem  cont/ol  (e.g.,  crash 
override,  barrier  arrest,  catapult  fire).* 

Numeric  keyboard.* 

IAS  control.* 

Me le  control. 

Selection  of  automated  training  exercises, 
checkrides,  and  demonstrations. 

Selection  of  types  of  normal  and  emergency 
procedures . 

Selection  ol  malfunctions  by  system. 

Selection  of  specific  pre  -det  erm  i  ned  rul- 
funct ions . 

Dynamic  replay  control. 

Print  control. 

Selection  ol  cockpit  instrument  monitor  ay 
cockpit  section. 

Selection  of  map  displays. 


VISUAL 

MONITOR 


DISPLAY  1 


DISPLAY  2 


1.  DRAINER  CONTROL 

2.  MODE  CONTROL 

3.  COMMUNICATIONS 

4.  AUTO  EXERCISES 

5.  DEMONSTRATIONS 

6.  PROCEDURES 

7.  MALFUNCTION  MENU 

8.  MALFUNCTIONS 

9.  VISUAL  SYSTEM  CONTROL 

10.  AIR  TURBULENCE  CONTROL 

11.  WIND  VELOCITY  CONTROL 

12.  TRIM  AIRCRAFT  MESSAGE 

13.  DIGITAL  CLOCK 

14  INStRUCTOR  CONTROL 
15.  MICROPHONE  PLUG  IN 


INSTRUCTOR 

AID 

SYSTEM 


Figure  2.  Generic  Instructor/Operator  Station  -  Alternative  2. 
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Other  controls  are  via  t lie  CRT  touch  panels, 
as  used  in  Alternative  1.  Typically  the  IOS 
console  touch  pads  are  used  to  access  the 
initial  oisplay.  Alternatives  emanating  from 
the  initial  display  are  typically  controlled 
from  touch  panels  on  the  faces  of  the  CRTs. 

The  identification  of  CRT  touch  panels  for 
each  display  is  based  on  an  analysis  of  the 
operations  which  instructors  may  choose  to 
exercise  at  a  given  point  i-  a  training  exer¬ 
cise.  Determination  of  display  content  is 
based  upon  requirements  to  monitor  trainee 
performance  ol  specitic  training  tasks  and  to 
control  the  training  exercise.  ‘these  ph'.los- 
t.phies  are  the  same  as  those  used  in  Alterna¬ 
tive  1.  They  are  designed  to  enab!"  instriu- 
‘ors  to  select  alternative  displays,  it. sort 
malfunctions,  mod i 1 y  weather,  etc.,  w  .hout 
having  to  carr.  out  a  multi-stop  selection 
process  through  a  series  oi  menus. 

Thee  are  a  number  of  other  fundamental 
ph  i  1  osoph  ies/ t  echn  i  ques  in  Alternative  d  which 
are  consistent  with  those  discussed  previously 
for  Alternative  !.  These  are  as  t allows: 


•  A  set  of  menus  and  selection  options  tor 
problem  control. 

•  Large  sets  of  pre-programmed  exercises  and 
initial  conditions. 

•  "Normal"  mode  of  operation  with  programmed 
exorcises  (i.e.,  automated  training  exer¬ 
cises). 

•  Extensive  instructor  prompting  and  pre¬ 
selected  options  in  the  free  flight  mode. 

•  Only  limited  capabilMy  to  make  on-line 
changes  to  initial  conditions  aad  pro¬ 
grammed  exercise  parameters. 

The  training  capabil’ties  of  bo*h  alterna¬ 
tives  are  the  same.  The  instructor  actions 
required  to  ‘Xercive  the  capabilities  ditfer, 
and  center  around  CRT  touch  panels  versus 
eonsol  e-mount  ed  electronic  tou-'it  pads*  It 
should  be  noted  that  since  CRT  touch  panels 
are  used  in  both  options  and  since  the  sott- 
vare  and  inst.  tu  tor  supperl  teatures  ire 
essentially  tin-  s..me,  many  of  I  he  operations 
are  the  same. 
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ADVANTAGES  AND  DISADVANTAGES 
Alternative  1 

Alternative  1  is  characterised  by  minimum 
use  of  fixed-function  electronic  touch  pads 
and  maximum  use  of  CRT  touch  panels.  The 
emphasis  is  on  simpliv.it>  of  the  IOS,  and 
simulation  monitoring  and  control  via  displays 
and  their  associated  touch  panels. 

Advantages .  As  a  result  of  the  low  number 
of  fixed-function  controls  on  the  IOS  console, 
the  IOS  is  simple  and  compact.  Also  contrib¬ 
uting  to  the  compact  design  is  the  clustering 
of  the  CRTs  into  what  is  essentially  a 
single-position  station.  The  appearance  of 
the  IOS  does  not  overwhelm  new  instr  actors. 
On  the  contrary,  it  presents  a  positive  image 
for  ease  of  operation. 

The  compact  size  of  the  IOS  enhances  the 
instructor's  span  of  control  and  enables  him 
to  easily  scan  all  controls  and  displays.  All 
operations  can  be  carried  out  while  the 
instructor  is  seated  at  a  single  position. 
The  station  is  designed  to  meet  human  factors 
specifications  for  roach  envelope,  instructor 
anthropometry,  visibility,  etc. 

Alternative  1  takes  full  advantage  of 
available  control,  display,  and  software 
state-of-the-art  technology.  Its  design  and 
operation  are  not  driven  by  the  technology. 
Rather,  they  are  intended  to  take  full  advan¬ 
tage  of  what  is  available  and  in  doing  so  to 
further  advance  the  state-of-the-art  in  IOS 
des ign. 

The  software  capacity  of  Alternative  1 
provides  for  maximum  flexibility  in  use  of  the 
trainer.  Controls  and  displays  are  readily 
accessible  through  the  software  associated 
with  the  displays  and  touch  panels.  Efforts 
have  been  made  to  use  concepts  which  minimize 
the  number  of  decision  steps  in  making  selec¬ 
tions.  This  results  in  increases  in  the  ease 
of  taking  advantage  of  the  design  flexibili¬ 
ties. 

Programmed  scenarios,  procedures,  alterna¬ 
tives,  e  t  • .  ,  are  readily  updated  through  soft¬ 
ware  changes.  This  flexibility  Is  afforded 
through  the  use  of  variable-function  touch 
panels,  as  opposed  to  f i xed-f unc t 1  on  touch 
pads,  and  the  use  oi  a  'v.riety  of  sottware 
support  features. 

Dlsadvant.ig.es.  Alternative  1  requires 
more  software.  This  greater  soitware  require¬ 
ment  in  comparison  to  Alternative  2  is  neces¬ 
sary  to  enable  scenario  selection  and  control 
through  a  central  master  menu. 

Mere  initial  iP  training  is  required  to 
allow  full  operat Ion  of  the  trainer.  This 
train'ng  primarily  involves  use  of  the  menus, 
faailiiritv  with  seleetion  alternatives,  and 
anticipation  ol  decision  points.  Through 
proper  programming  and  use  of  the  IAS,  and  a 


limited  amount  of  experience,  it  is  antici¬ 
pated  that  this  disadvantage  would  be  mini¬ 
mized  . 

In  certain  situations  more  steps  are 
required  to  select  desired  displays  and  .nake 
data  entries.  The  steps  are  required  to 
branch  through  the  menus  and  indices  to  access 
the  desired  end  point. 

Alternative  2 

Alternative  2  is  characterized  by  exten¬ 
sive  use  of  fixed-function  electronic  touch 
pads  on  the  IOS  console.  The  touch  pads  serve 
the  same  functions  as  many  of  the  menus, 
indices,  and  touch  panel  selection  options  ’’n 
Alternative  1.  The  major  IOS  configuration 
difference  is  the  prevalence  of  these  elec¬ 
tronic  touch  pads. 

Advantages.  Most  of  the  basic  simulation 
controls  are  located  on  the  IOS  console.  They 
are,  therefore,  always  available  for  observa¬ 
tion,  study,  and  activation.  Through  their 
presence  and  labelling  many  of  the  capabili¬ 
ties  of  the  device  are  displayed  without 
reference  to  manuals  or  the  software. 

The  software  required  in  Alternative  2  is 
loss  complex  than  that  in  Alternative  1.  This 
results  from  the  use  ol  fixed-function  elec¬ 
tronic  touch  pads  on  the  IOS  console  in  lieu 
of  menus,  indices,  and  CRT  touch  panels  which 
are  used  in  Alternative  1. 

Since  many  of  the  device  controls  and 
capabilities  are  "hard-wired"  in  .he  IOS 
console,  ID  skill  and  training  r«.-qu  i  remen  t  s 
are  reduced.  It  is  anticipated  that  IOS 
familiarization  training  would  be  simplified, 
retention  improved,  and  maintenance  and 
upgrade  of  skills  improved. 

The  use  of  * i xcdef unc t I . n  electronic  touch 
pads,  in  many  ca:.es,  minimizes  the  nurber  or 
steps  required  to  access  controls  and  dis¬ 
plays.  'Hie  touch  pads  enable  the  instructor 
to  go  directly  to  the  final  se'ecMnn  index  or 
the  desired  end  display  without  branching 
through  a  Sequential  set  ot  menus.  Thus 
efficiency  is  enhanced. 

Pi  sadvant ages .  Alternative  2  contains  a 
largo  number  ot  J  i xed- 1 unc t ion  electronic 
touch  nads  on  the  IOS  console.  Even  with  a 
layout  based  on  good  principles  ot  human  fac¬ 
tors,  (lie  JOS  will  appear  cluttered  and  nay  in 
certain  situations  he  ineflicient  to  operate 
or  lead  to  contusion.  The  clutter  have  a 

psycho 1 og ion  1  impact  on  instructors  ( i .e. ,  tin- 
complexity  ot  the  device  operations  n..y  appear 
overwhelm! ng) . 

To  acconnoda t e  the  layout  ot  console- 
mounted  electronic  touch  pads,  the  IOS  will  he 
larger  than  that  in  Alternative  1.  This 
increased  s  •'  e  nay  a!  lent  the  instructor's 
span  of  control  and  ability  to  scan  the  IOS 
controls  and  displays.  It  may  also  reduc* 


the  instructor's  capability  to  conveniently 
control  all  operations  from  a  seated  position. 

The  greater  use  of  fixed-function  controls 
rather  than  software  ariven  variable-function 
panels  makes  revision  and  update  more  diffi¬ 
cult.  Some  revisions/updates  will  require 
hardware  changes.  These  same  revisions/ 
updates  in  Alternative  1  would  require  only 
software  changes. 

Control  of  exercises  is  divided  between 
two  modes:  panel-mounted  electronic  touch 

pads  and  CRT  touch  panels.  It  is  anticipated 
that  operations  using  the  two  modes  will  be 

less  efficient  than  the  same  operations  using 
a  single  mode  (e.g.,  CRT  touch  panels  in 

Alternative  1). 

CONCLUSIONS 

IOS  design  has  suffered  from  the  same 

malady  that  has  plagued  the  design  of  many 
other  pieces  of  sophisticated  hardware: 
insufficient  analysis  efforts  and  data  on 
which  to  base  decisions.  As  a  result,  many 
previous  lOSs  have  not  been  designed  to  facil¬ 
itate  training.  They  have  contained  the 
shortcomings  discussed  in  this  paper. 

The  primary  conclusion  ol  this  paper  is 

that  the  design  of  an  efficient,  functional 
IOS  requires  systematic  application  of  a 
variety  of  human  factors  and  training  data. 
The  alternatives  presented  previouslv  resulted 
from  such  a  process.  Future  IOSs  should  he 
deiived  through  a  similar  analysis  process  In 
which  candidate  JOS  features  are  systemati¬ 
cally  identified  and  assessed.  Selection  of 
features,  operating  procedures,  and  configura¬ 
tions  should  be  based  on  ultimate  goals  . 
optimizing  both  instructor  performance  and 
overall  simulator  training  effectiveness.  The 
resulting  usability  of  the  device  will  provide 
for  increased  user  acceptance  and  improved 
training  system  effectiveness. 

Future  IOS  design  improvement  efforts 
should  focus  on  expansion  of  the  recommenda¬ 
tions  presented  in  this  paper.  Possible 
topics  include  (1)  applications,  limitations 
and  values  of  specific  instructional  features 
(e.g.,  playback,  performance  measurement,  and 
demonstrations),  (2)  refinement  of  the  design 
principles  tnd  concepts,  (3)  extension  of  the 
design  principles  and  concepts  to  other  types 
of  simulators,  and  (A)  modelling  the  IOS 
design  process. 
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ABSTRACT 


If  logistics  concepts  are  to  maintain  pace  with  the  constantly  changing  hardware  and 
software  spheres,  innovative  state  of  the  art  logistics  acquisition  strategies  must  be 
conceived,  defined,  and  implemented.  Logistic  acquisition  concepts  in  the  past  have 
served  well  those  devices  for  whose  support  they  were  desigred.  They  cannot,  however, 
neet  the  challenges  occasioned  by  rising  costs,  the  increasing  demand  for  more  reliable 
life  cycle  support,  and  ever  more  sophisticated  training  device  systems.  These  challenges, 
as  well  as  others,  can  best  be  met  by  developing  a  best  value,  performance  based,  func¬ 
tionally  oriented  logistics  acquisition  strategy.  Such  an  evolution  will  focus  attention 
on  the  product,  not  the  process,  of  the  logistics  support  acquisition  effort.  In  turn, 
this  shift  in  focus  will  require  botu  Government  and  contractor  to  relinqu'-h  certain 
traditional  roles  and  to  assume  some  unfamiliar  responsibilities.  The  authors  recognize 
that  such  an  approach  will  cause  some  degree  of  discomfort  to  both  Government  and  con- 
tract?r  personnel.  However,  they  submit  that  just  as  technology  has  advanced  from  the 
vacuum  tube  tc  the  integrated  circuit,  so  must>logisticsN^dvance  from  emphasis  on  the 


acquisition  process  to  concentration  on  acquir 
reliable  life  cycle  support  to  training  device 


Changes  to  our  current  procedures  for  pro¬ 
viding  training  device  support  are  being  mandated 
bv  three  persistent  and  omnipresent  forces:  (1) 
the  cascading  flood  cf  technological  advancements 
in  all  pha.es  of  electronics,  (0)  the  escalating 
costs  and  decreasing  quality  of  logistics  support 
efforts,  and  (3)  the  unpredicable  availability  of 
technical  personnel.  Although  the  necessity  for 
modernizing  logistics  support  procurement  methed- 
has  been  recognized  for  years,  few  methodological 
improvements  have  been  planned,  designed,  or  im¬ 
plemented.  The  logistics  community  has  far  too 
long  focused  on  the  process,  not  the  product  of 
the  logistics  prccuremert  effort.  As  a  result, 
we  find  ourselves  mere  and  more  involved  in  a 
game  of  "catch  up'*  rather  than  in  a  role  of 
leadership.  Thus  engaged,  we  have  failed  to 
assert  the  vital  importance  of  front  end  planning 
for  logistics  support.  The  inevitable  result  has 
been  the  generation  of  conflict  between  "what  is" 
anc  "what  should  be"  in  terms  of  training  device 
and  simulator  support. 

The  pressure  ofn? rated  by  this  conflict  is 
influencing  people  «•*  removed  from  our  situation 
to  enter  the  arena  loaistics  decision  makers. 


rg  logistics  products  which  wnl  provide 
systems* 

These  people,  who  are  located  in  all  strata  of 
responsibility,  from  training  device  user  tc  tie 
Secretary  of  the  Nav> ,  have  a  vested  irterest  in 
the  availability  of  training  device  hours,  in  the 
cost,  and  effectiveness  required  to  provide  this 
availability.  They  are  telling  us  that  our  tradi¬ 
tional  procurement  philosophies  are  no  longer 
meeting  their  needs.  Receiving  little,  if  any, 
positive  response  from  the  logistics  community, 
they  perceive  that  such  inaction  indicates  a  lack 
of  ability.  Because  of  this  cormunication  gap,  we 
now  stand  on  the  precipice  of  being  told  that  a 
cure  will  be  effected  by  eliminating  organic  sup¬ 
port  in  favor  of  full  contractor  support. 

Before  discussing  the  changes  that  are  re¬ 
quired  to  upgrade  our  logistics  acquisition 
strategies,  it  is  imperative  that  we  gain  some  in¬ 
sight  into  the  conditions  which  have  led  to  this 
need.  Only  through  an  understanding  of  the  cur¬ 
rent  situation  can  we  appreciate  the  need  for  a 
logistics  acquisition  strategy  which  will  control 
costs,  provide  more  reliable  support,  keep  pace 
with  technology,  and  provide  the  proper  foundation 
for  organic  or  transition  to  full  contractor 
support  in  the  future. 
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It  may  be  helpful  to  realize  that  this  need 
is  not  unique  to  the  training  device  and  simu¬ 
lator  environment.  A  recent  Society  of  Logistic 
Engineers  Newsletter  contains  extracts  from  an 
Acquisition  Improvement  Task  Force  Report  on  the 
Carluccl  initiatives.  (!)  This  report  sums  up 
three  areas  of  failure  in  support  and  readiness 
throughout  the  Defense  Department  acquisition 
process.  First,  and  well  known  to  most  of  us  in 
the  logistics  business  is  the  lack  of  management 
emphasis  and  support  given  to  logistics.  The 
report  states  that  "support  is  still  running  a 
poor  fourth,  after  performance,  schedule,  and 
cost."  Next,  the  report  cites  a  lack  of  front 
end  logistics  analysis  as  being  responsible  for 
building  in  higher  support  costs  f rent)  the  very 
beginning.  Finally,  the  report  states  that  sup¬ 
port  areas  suffer  from  inadequate  procedures, 
organization,  and  tec.nical  capability.  These 
weaknesses  are  aggravated  in  the  training  device 
world  through  a  combination  of  a  dinosaur  like 
devotion  to  tradition  and  rigid  Government  pro¬ 
cedures  which  govern  almost  all  aspects  of  a 
training  device  procurement. 

The  traditional  tendency  for  low  risk  pro¬ 
curements  and  the  lack  of  flexibility  in  logis¬ 
tics  acquisition  procedures  have  been  addressed 
at  previous  industry  conferences  and  in  other 
areas  of  the  acquisition  literature.  However, 
such  treatments  have  tended  to  stress  fhe  theo¬ 
retical  aspects  of  the  problems  using  a  broad 
brush  approach.  And  regrettably,  seldom  has  a 
positive,  feasible  alternative  been  offered  as  a 
possible  solution  to  both  present  and  anticipated 
problems. 

It  appears  valid  to  state  that  we  car.  no 
longer  continue  to  do  business  as  usual  If  we 
are  to  I.eep  pace  with  the  technology  we  are 
responsible  for  supporting.  We  can  no  longer 
continue  to  stress  the  theoretical  aspects  of 
logistics  while  giving  scart  attention  to  the 
dynamics  of  the  market  place.  We  can  no  longer 
continue  to  loosely  apply  the  concept  of  inte¬ 
grated  logistics  support,  while  we  allow  each 
logistic  specialty  to  go  off  ir.  divergent 
directions.  We  can  no  longer  force  logistic* 
and  program  managers  into  the  role  of  simply 
coordinating  between  individual  logistics 
specialties  such  as  training,  provisioning,  pub¬ 
lications,  and  others.  We  must  provide  our 
managers  with  the  tools  required  to  overcome 
these  shortcomings. 

But  such  tools  will  be  forged  only  when  we 
recognize  that  logistics  planning  must  begin  with 
and  be  accorded  the  sate  stature  as  functional 
descriptions,  niliUry  charac '.eristics ,  ergi- 
nterirg  specification*  ard  other  docurtrls  which 
are  drawr  up  for  tk.  hardware  and  software.  Only 
then  will  we  be  able  to  adopt  and  adhere  to  a 
state  of  the  art,  innovative  logistics  support 
acquisition  strategy  which  is  a  truly  Integrated 
package.  Our  recommendations  foi  ac  :cnpl  ishir.g 
this  start  with  a  review  oh  Department  cf 


Defense  (DOD)  Directive  5000.1  of  March  1982, 
entitled  Major  Systems  Acquisition,  wrich  states 
that  a  cost  effective  balance  must  be  achieved 
among  acquisition  costs,  system  owr.arship  costs, 
and  systems  effectiveness  in  terms  of  the  mission 
to  be  performed.  The  directive  goes  on  to 
say  that  the  acquisition  strategy  developed  for 
each  system  should  consider  the  unique  circum¬ 
stances  of  individual  programs  and  that  these- 
programs  shall  be  executed  with  innovation  and 
common  sense. 

It  is  realized  that  Department  of  Defense 
promulgated  this  directive  for  major  systems 
acquisition  and  that  training  devices  are  seldom 
considered  major  systems.  Still  the  logic  is  no 
less  valid  for  the  procurement  of  training  devices 
and  simulators,  particularly  if  a  series  of 
devices  is  being  contemplated.  Innovation  and 
conmon  sense  must  be  the  prlne  requirement  in  oil 
areas,  including  logistics.  What  is  a  good  logis¬ 
tics  strategy  for  one  device  is  not  necessarily 
appropriate  for  another  device.  A  logistics  pro¬ 
curement  strategy  developed  for  a  series  of  four, 
five,  six  or  more  devices  is  not  appropriate  for 
a  one  or  two  device  buy.  Obviously,  the  reverse 
holds  true. 

It  is  the  responsibility  of  both  Government 
and  contractor  personnel  to  recognize  this  and  to 
develop  unique  procedures  for  each  buy.  The  day 
of  the  boiler  plate,  cut  and  paste  logistic  pack¬ 
age  is  long  past.  We  believe  that  what  is 
required  in  today's  environment  is  the  develop¬ 
ment  and  implerer.taticn  of  a  state  of  the  art 
logistics  support  acquisition  strategy  based  cn 
the  guidance  of  DOD  Directive  50C0.1  and  on  the 
guidance  of  ether  directives  such  as  the  Defense 
Acquisition  Regulation  (3)  which  sets  *ortk  tKe" 
basic'  1  egaYYfcctr’i ne  defining  the  role  of  Gevcrr- 
nent  ar.d  contractor  relationships;  Office  of 
Management  and  Budget  (CtiC)  Circular 7.-TC$  W 
which  TurtTTer  dcTines  these  rcYes  cr.d  explicitly 
recommends  that  the  Government  establish  syster 
performance  objectives  and  allow  the  contractor 
to  develop  dttailed  design  objectives.  These 
recomendations  *re  further  defined,  enumerated 
ir,  0MB  Circular  A- 7b,  which  outlines  the 
Office  of  Management  and  Budget's  pcllries  cf 
acquiring  commercial  prodicts  and  services  for 
Gcvcrrricnt  use.  fisc  germane  i:  the  Secretary  of 
the  Navy  (SCCNAV)  Inn r uct  1 r  r_JS 2C_0. ?] A ,  (6)  wnTcri 
cTearTy  defines  the  rcTt  of 'contractor  personnel . 
These  directives  and  policy  statements  provide 
the  cor  nerstor.es  fer  our  recorrcrdaticns. 

If  loqistics  personnel  are  to  cvercor>e  their 
current  problems  end  ret.. in  ccrtrcl  over  their 
own  future,  drastic  charges  ir.  thinking  are  re¬ 
quired.  Today's  logistics  requirements  call  for 
reliable,  cost  effective  life  cycle  support  o* 
the  highly  sophisticated  and  dyr&r.ic  hardware  erd 
software  subsystems  which  constitute  ^  tralnirg 
device.  To  meet  these  requirements,  the  authors 
advocate  a  performance  based,  'urcticndly 
Drier. ud  logistics  procurement  approach  that 
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focuses  attention  on  the  product,  not  the 
process,  of  the  logistics  support  acquisition 
effort.  This  strategy  is  defined  as  a  flexible, 
product  based  approach  tc  procurement  of  logis¬ 
tics  material  and  services  which  rests  on  the 
assumption  that  the  device  contractor  is  in  the 
best  position  to  determine  the  most  cost  effec¬ 
tive  and  efficient  method  for  ievelopment  of  the 
life  cycle  support  for  that  device. 

The  Performance  Based,  Functionally 
Oriented  approach  differs  from  the  present 
approach  in  two  fundamental  concepts.  First  is 
the  concept  that  the  only  measure  of  satisfactory 
logistics  performance  is  the  effectiveness  of  the 
final  logistics  package  in  supporting  the  device. 
Time  and  effort  should  r.o  longer  be  wasted  in 
scrutinizing  every  aspect  of  the  development 
schedule,  in  developing  detailed  plans  for  every 
conceivable  element,  in  requiring  reports  for 
every  contractor  movement,  for  developing  and 
maintaining  detailed  milestone  charts,  etcetera. 
These  are  tasks  which  ere  functions  of  the  con¬ 
tractor  and  which  the  contractor  must  do  if  he 
is  to  raet  his  contractual  requirements  with  an 
acceptable  product.  Government  personnel  should 
focus  on  the  quality  of  the  delivered  product. 
Second  is  the  idea  that  performance  of  the  device 
for  the  10  or  15  years  it  will  be  utilized  is  of 
at  least  equal  importance  to  the  3  to  5  years  it 
is  in  the  development  and  manufacturing  stages; 
thus,  logistics  considerations  should  be  given 
?qual  weight  in  the  request  for  proposal  and  in 
che  evaluation  and  payment  clauses  of  the 
contract. 

From  these  fundamental  corcepts  arc  derived 
three  basic  premises: 

1.  logistics  support  is  an 
integral  part  of  device  procurement 
and  the  contractor  should  be  tasked 
with  the  full  ar.d  tctal  responsi¬ 
bility  cf  providing  a  reliable  and 
cost  effective  logistics  support 
package. 

2.  With  the  proper  incentive, 
contractors  will  respond  to  Govern¬ 
ment's  logistics  needs. 

3.  Government  can  provide 
this  incentive  by  stipulating  in 
the  contract  that  a  najor  amount 
of  total  payment  will  be  withheld 
urtil  logistic  acceptance  criteria 
are  rret. 

These  concepts  are  not  new,  but  they  have 
beer  neglected  ir  the  quest  fcr  sophistication. 
What  , ay  be  unfamiliar  is  that  the  Performance 
Eased,  Functionally  Oriented  concept  is  based  or 
the  premise  that  if  the  Government  provides  the 
logistics  requirements,  establ.shes  neasurenents 
to  determine  if  these  requ-i rervrts  have  been  net, 
and  by  using  the  common  sense  mentioned  in  TOC 
Pirective  S00C.1,  refust>  to  pay  for  rati  rials 
arc  services  that  do  not  meet  the  established 
criteria,  cer*i’a*tusl  cost  effective  reliable 


life  cycle  logistic  support  will  be  forthcoming. 
Industry  has  the  capability  to  respond;  the 
Government  needs  only  to  develop  and  provide  the 
proper  stimulus. 

Logistic  support  requirements  currently  vary 
from  full  organic  support  to  full  contractor 
support.  The  Performance  Based,  Functionally 
Oriented  logistics  acquisition  strategy  was  devel¬ 
oped  to  provide  a  flexible,  non-complex  method  of 
satisfying  varying  logistics  requirements.  The 
Performance  Based,  Functionally  Oriented  concept 
provides  the  contractor  an  opportunity  to  utilize 
his  creativity,  technical  expertise  and  common 
sense  to  develop  an  effective  support  program 
when  given  an  established  set  of  objectives  in  the 
logistic  statement  of  work.  The  end  result 
expected  from  the  contractor  is  a  cost  effective 
life  cycle  support  program  which  can  be  verified 
by  the  Government  through  evaluations  of  the  per¬ 
formance  and  usefulness  of  people,  equipment,  and 
documentation  against  a  set  of  measurable 
objectives. 

The  Performance  Based,  Functionally  Oriented 
acquisition  strategy  consists  of  three  phases: 
definition,  performance,  and  acceptance.  Durirg 
the  definiticn  phase,  the  Government  integrated 
logistic  support  team  joins  the  engineering  tean 
in  conducting  a  front  end  analysis.  In  con¬ 
junction  with  the  engineering  team  and  with 
project  management,  a  firm  maintenance  concept  is 
decided  upon.  Using  this  maintenance  concep*  as 
the  '.eystone  of  all  future  integrated  logistic 
support  planning,  the  integrated  logistic  support 
team  works  together  to  develop  ar  integrated  log¬ 
istic  statement  of  work  that  clearly  delineates 
the  aims,  goals,  and  objectives  that  must  be 
achie.-ed  if  the  required  level  of  life  cycle  sup¬ 
port  is  to  be  realized.  Tc  properly  carry  out  the 
all  important  task  of  developing  a  logistic  state¬ 
ment  of  w*.  rk  that  will  contain  all  the  necessary 
criteria  for  performance  n-easurement  will  n?qu*re 
a  major  effort  from  each  men  tor  ce  the  integrated 
logistic  support  team.  Each  subsystem  of  the 
device  must  be  analyzed  individually  and  ,'<■  part 
cf  t*e  overall  system.  The  integrand  legis*. 
support  team  will  be  required  to  thoroughly 
research  all  requi  rer-onts  arid  ensure  !h*t  t  he 
criteria  developed  are  needed,  attainable, 
measurable ,  and  comprehensive.  Free  this  is  cere, 
the  logistic  statement  cf  vo»k  beeves  the  driving 
force  for  the  remainder  of  the  • or*  r  act . 

WMle  the  definUicn  phase  is  primarily  the 
i  esponsibf'Hty  cf  the  Government,  the  performance 
phase  berimes  almost  totally  the  contractor's 
rtsforsibi 1 i ty.  Grce  the  contractor  receives  the 
logistic  statement  of  work ,  it  is  up  u  h|>  tc 
develop  r»n  integrated  legist Hs  support  grogr>r 
uv’ng  state-of- the- art  techniques.  Cere  \he  con¬ 
tractor  dcvelcps  and  receives  approval  or.  t  In 
integrated  logistic  support  pW,  he  ►ill  1  'vt  re 
further  reviews,  approvals,  or  other  inter fer ere e 
ft  os*  Government  personnel,  looking  at  this  f»cr 
another  perspective,  he  will  no  longer  b«  -tbit  tc 
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depend  on  Government  specialists  to  perform  a 
large  share  of  Ms  work  for  him.  Here,  the 
Government  should  adhere  strictly  to  the  pro¬ 
visions  of  SECNAVINST  4200. 27A. 

The  last  phase,  the  acceptance  phase,  once 
again  becor.^s  the  responsibility  of  the  Govern¬ 
ment.  Through  a  series  of  verifications  and 
demonstrations ,  the  Government  will  verify  the 
contractor  delivered  product  against  the  criteria 
set  forth  in  the  logistic  statement  cf  wort  and 
approved  in  the  ir  roted  logistic  support  plan. 
DEFINITION  PHeSE: 

During'This  1  *  . a  1  phase,  the  Gcvernnent 
integrated  logistic  support  team  n-embers  must 
engage  in  conducting  a  thorough  front  end  logis¬ 
tics  analysis.  The  time  is  past  when  logistics 
personnel  could  waU  until  the  device  became  a 
conceptual  reality  end  then  build  a  support  pro- 
gran  around  it.  In  addition  to  determining  the 
logistics  needs,  the  integrated  logistic  support 
team  members  must  become  technical  advisors  on 
all  committees  and  ether  teams  connected  with 
developing  front  end  documents.  Such  actions 
will  ensure  that  logistics  become  an  integral 
part  of  the  device  procurement. 

Once  the  front  end  analysis  is  accomplished, 
the  integrated  logistic  support  team  must  utilize 
the  logistic  statemert  of  work  developed  as  a 
result  of  that  analysis  to  prepare  a  request  for 
proposal.  The  total  '"tegruted  logistic  support 
team  should  become  involved  in  *he  development 
of  the  request  for  proposal  to  ensure  that  the 
package  is  totally  integrated  and  supportable. 

In  developing  the  inputs  the  team  members  must 
take  into  account  the  proposed  training  device, 
who  is  going  to  man  the  device  as  instructors, 
operators,  maintenance  personnel  and  above  all, 
what  is  the  maintenance  concept/plcn  for  the 
device.  With  these  facts  in  mind  the  logistic 
tear  develops  a  list  of  terminal  objectives  out¬ 
lined  ir.  a  logistic  staterent  of  work  which 
ensures  that  the  device  '..'ill  be  supportable  tc 
tn«  required  maintenance  plan  level.  Next,  the 


team  develops 
inputs : 

the 

following  procurement  package 

i  „ 

Funding  Pequirer.«nts. 

2.* 

P.eqccs*  for  Proposal. 

a. 

Schedule. 

b. 

Final  Logistics  Stalwart 
of  Won.. 

c. 

(DD  Form-  1423)  Integrated 
Logistics  Su| oort  Plan 
Requirements. 

d. 

Technical  Proposal 
Requirements. 

e. 

Proposal  Evaluation  Plan. 

The  technical  proposal  requirements  take  on 
a  new  meaning  under  the  Performance  Based, 
Functionally  Oriented  Program  because  the  Govern¬ 
ment  is  rc  longer  evaluating  proposals  baTed  on 
yes/nc  responses.  The  contractor’s  proposal 
evaluation  is  based  on  how  well  he  respond*  to  an 
established  criteria  by  applying  st^to-of-ihe-art 


prcced"res.  Mo  longer  will  the  contractor  hrve 
a  boiler  plate  set  of  guidelines  that  he  can  feed 
back  to  the  Government  es  a  logistics  support 
outline  in  order  to  receive  a  satisfactory  pro¬ 
posal  evaluation  rating.  He  must  use  his  own 
skills,  abilities,  ard  techniques  to  determine 
the  best  approach  ir  oevelcping  a  logistic  support 
program  that  is  totally  integrated  anc  which  will 
support  the  device  outing  its  life  cycle. 

The  only  logistic  docurent  required  by  the 
Contract  Data  Requirements  List,  DD  Form  1423, 
during  the  request  for  proposal  is  the  integrated 
logistic  support  plan.  The  integrated  logistic 
support  plan  will  outline  the  contractor's  log's- 
tic  support  plan  for  the  devW  and,  anong  other 
items,  will  contain  the  contractor  s  recommenda¬ 
tions  for  all  c'h-e**  logistic  documentation. 

Proposal  evaluation  procedures  will  require 
a  highly  skilled  professional  logistics  team  with 
expertise  ir  log.ji.ics  management,  the  technical 
aspects  of  the  trainer,  a^d  in  specific  areas  cf 
logistics.  The  evaluation  will  be  acconpl ished 
in  two  parts:  *it$i,  by  a  group  of  highly 
qualified  technical  specialists-  and  s. condly,  by 
a  joint  logistic  team  tc  ensure  that  each  element 
is  integrated  intc  a  total  logistic  effort.  Tits 
joint  evaluation  will  assist  the  negotiation 
efforts  and  ensure  a  two  party  win-wir,  situatior 
that  rill  help  guarantee  a  quality  product  for 
the  fleet. 

PERFORMANCE  PHASE : 

"The  first  step  after  contract  aw^rd  will  te 
a  scries  of  conferences  between  the  Government 
logistics  team  ard  the  contractor  to  define  the 
detailed  requirements  of  the  integrated  logistic 
support  plan.  The  primary  reason  fov”  the  Perfor¬ 
mance  Based,  Functionally  Oriented  Program  is  tc 
create  an  innovative,  flexible  integrated  logis¬ 
tic  support  system.  Under  this  concept,  the 
integrated  logistic  suppori  plan  becomes  the  key- 
stcr.e  document  which  is  meant  to  ensure  that  eacf 
training  device  is  provided  a  quality  support 
program.  No  longer  will  each  specialist  have 
his/her  own  management  document  governing  a 
specific  logistics  aroa.  All  areas  will  be 
governed  by  the  integrated  logistic  support  plan. 
The  integrated  logistic  support  plan  will  be 
broken  down  into  a  specific  set  of  sections  which 
will  be  determined  by  the  needs  of  the  device. 
Specific  sections  that  nay  be  required  are: 

1.  Introduction. 

2.  Management. 

3.  Maintenance  Pian/Concept . 

4.  Personnel  and  Training. 

5.  Logistic  Data. 

6.  Support  and  Tes*  Equipment. 

2.  Supply  Support. 

8.  Facilities. 

9.  Interim  Support. 

10.  Transportation  Handling. 

11.  Integrated  Logistic  Support 
Accountabi li ty. 

12.  Verification  Demonstration  Plan, 


Sections  2,  11,  and  12  will  be  disclosed 
briefly  since  they  will  contain  the  specific 
requirements  which  are  the  essence  of  a  success¬ 
ful  integrated  logistic  support  program. 

Section  2,  Management:  This  section  should  spell 
out  the  integrated  logistic  support  management 
program  3nd  how  each  logistic  element  will  be 
integrated  into  the  total  support  program  to 
ensure  that  the  logistic  services  provided  are 
all  to  the  same  level.  Section  1 i ,  Integrated 
Loglsti c  Support  Accountability:  TKTs" section 
should  be  a  plan  spelling* out  how  the  members  of 
the  device  team  are  going  to  account  for  specific 
items  and  funds  to  ensure  that  specific  alloca¬ 
tions  are  slated  for  a  specific  area  and  not 
diverted  to  another  element  without  coordination 
with  all  concerned.  Section  12,  Verification 
Demonstration  Program:  This  section  should”' 
define  how  the  Government  is  going  to  verify  the 
contractor's  perform, ance. 

Each  section  of  the  integrated  logistic 
support  plan  will  be  a  stand  alone  document  for 
the  management  of  the  individual  elements.  When 
the  sections  *re  combined,  they  will  form  a 
volume  which  w r 1 1  provide  a  comprehensive  inte¬ 
grated  logistic  support  plan  for  the  logistic 
manager.  Each  logistics  specialist  will  be 
responsible  for  the  development  and  acceptance 
of  his/her  specific  section(s)  within  the  inte¬ 
grated  logistic  support  plan.  With  this  method 
of  developing  the  integrated  logistic  support 
plan,  the  logistics  specialists  will  have  the 
flexibility  to  manage  their  own  specific  areas 
and  yet  give  the  logistic  manager  the  capability 
cf  total  integrated  logistic  support  management. 

During  the  orientation  conference,  the 
Government  will  outline  its  Interpretation  of 
the  contractor's  proposal  as  it  relates  to  the 
aims,  goals ,  and  objectives  of  the  logistic 
statement  cx  work.  These  differences  will  be 
reconciled  during  the  conference.  Upon  com¬ 
pletion  of  the  conference,  the  contractor  vp  1 1 
forward  the  revised  integrated  logistic  support 
pi ar  tc  the  Goverrmert  for  review  prior  to  the 
integrated  logistic  support  plan  conference. 
~hirty  days  after  delivery  of  the  preliminary 
integrated  logistic  support  plan,  an  integrated 
logistic  support  plan  conference  will  be  con¬ 
ducted.  Curing  the  integrated  logistic  support 
plan  conference,  the  contractor  will  define,  in 
detail,  his  concept  of  the  logistic  prograr  as 
stated  ir  the  integrated  logistic  support  plan, 
fit  this  time  the  integrated  logistic  support  plan 
will  be  accepted  and  tecome  the  prime  logistic 
support  document  'or  the  life  cf  the  training 
device. 

The  function  of  the  contractor  during  the 
performance  phase  cf  the  device  is  based  on  the 
contractual  agreements  defined  in  '"ho  integrated 
logistic  support  plan.  These  functions  will  vary 
depending  on  the  maintenance  ccncept/plan,  Mze 
cf  the  device,  I'SEP's,  and  type  c-f  trc’ner.  Cnee 
the  integrated  logistic  support  plan  has  been 


approved,  the  contractor  assumes  full  responsi¬ 
bility  for  development  of  the  logistics  support 
program. 

ACCEPTANCE  PHASE: 

The  acceptance  phase  is  the  Government's 
responsibility  in  that  the  Government  must  estab¬ 
lish  the  criteria  against  which  the  contractor's 
performance  will  be  judged.  These  expectations 
must  be  stated  in  terms  cf  performance  based, 
functionally  oriented  aims,  goals,  and  objectives. 
This  is  accomplished  as  described  in  the  defini¬ 
tion  phase  above. 

The  evaluation  method  utilized  to  certify 
program  acceptability  wil1  necessarily  vary  from 
project  to  project  since  it  is  dependent  upon  the 
outcome  cf  the  front  end  analysis  accomplished  for 
each  device.  However,  in  all  cases,  the  methods 
chosen  will  evaluate:  (1)  the  ability  of  the 
personnel  assigred  to  support  the  device  to 
properly  fulfill  their  responsibilities;  (2)  the 
usefulness  of  the  support  equipment  such  as  tools, 
test  equipment,  automatic  test  equipment  and 
diagnostics  in  enabling  maintenance  personnel  to 
carry  out  the  designated  level  of  maintenance;  and 
(3)  the  accuracy,  quality  and  quantity  of  docu- 
n-ents  such  as  maintenance  publications,  vendor 
r'anuo!s,  PMS  docunentation,  training  publications 
and  other  required  drawings,  documentations  and 
publications. 

Since  the  contractor  has  been  assigned  total 
responsibility  for  development  of  the  total 
logisfics  support  prograr-  during  the  performance 
phase,  he  is  totally  responsible  for  the  accept¬ 
ability  of  his  delivered  product.  As  stated 
earlier,  the  provisions  of  SECNAVINST  4200. 27A 
should  be  taken  literally  and  applied  unequivo- 
cably : 

The  Government  may  .  .  .  obtain  the 
(required)  work  by  contract,  providing 
two  conditions  are  met:  (1)  the  con¬ 
tract  itself  must  ask  for  the  finished 
product,  only,  and  (2)  the  contract 
must  be  administered  ir  such  a  way  that 
control  and  supervision  ever  the  work 
and  dis-.retion  as  to  the  techniques 
which  will  be  user  remain  solely  with 
the  contractor.  .  .  .  The  intent  of 
this  statement  is  made  explicit  in 
the  next  sentence.  ...  In  ether 
words,  if  the  Gove.nment  wants  a 
building  painted  it  defines  the  job, 
lets  the  contra  tor  paint  the  building 
as  he  sees  fit,  and  then  accepts  it 
or  rejects  it  solely  on  the  basis  of 
whether  the  cor.pl eted  job  meets  the 
contact  specification.  .  .  . 

Thus,  if  the  contractor  has  not  lived  up  to 
his  contractual  agreements,  he  alone  will  be 
financially  responsible.  He  will  be  assessed  a 
significant  portion  cf  the  overall  ccr tract  amount 
for  his  failure  to  perform.  This  nssessnent  may 
then  be  used  by  the  Government  to  correct  defi¬ 
cient  areas  through  other  means  c*  to  obtain, 
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through  agree  lent  with  the  contractor,  extended 
contractor  support,  until  the  contractor  c-n 
correct  his  own  deficiencies.  The  authors 
believe  that,  since  a  training  device  which  lacks 
satisfactory  support  is  no  more  useful  than  a 
library  without  books,  at  least  5C  percent  of 
the  total  contract  price  should  be  assessed  fcr 
failure  to  provide  a  satisfactory  logistics 
support,  program. 

SUMMARY 

The  rising  costs,  decreasing  quality  and 
flexible  method  of  the  cu  rent  procedures  for 
procuring  logistics  support  for  training  devices 
have  made  changes  inevitable.  This  paper  has 
•ddtessed  several  of  these  problems  ur.J  has 
pointed  out  why  fresh  thinking  is  needed  now. 
Obviously,  there  •“  no  perfect  solution  to  the 
multifaceted  prob.ems  associated  with  providing 
reliable,  cost  effective  life  cycle  support  to 
training  devices  and  simulators.  The  authors 
have  proposed  the  Performance  Based,  Functionally 
Oriented  Logistics  Acquisition  Strategy  as  a 
pcssiole  framework  wh'ch  can  be  emh  lished  upon 
to  achieve  two  important  goals.  The  first  is  the 
immediate  need  tc  improve  c-r  logistic  acqui¬ 
sition  policies.  The  second  is  tr  provide  a 
smooth  transitional  step  into  ’he  e~a  cf  full 
cortractor  support,  which  tht  authors  believe  is 
both  inevitable  and  desirable  jiven  the  pre¬ 
dictable  events  cf  the  future. 
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SOME  ANALYTICAL  THOUGHTS  ON  ONE  ANSWER  TO  THE  ARMED  SERVICES 


TRAINING  MANPOWER  CRUNCH  -  TRAINING  BY  CONTRACTORS 


Dr.  Jack  Ballard 
Martin  Marietta 
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ABSTRACT 


Numerous  studies  and  discussion  sessions  have  focused  on  the  military  personnel  shortages 
extending  through  the  year  2000.  The  manpower  problem  promises  to  be  acute  according  to  de¬ 
mographers,  as  government,  industry,  and  the  military  compete  for  a  share  of  the  shrinking  pool 
of  young  work-age  personnel.  Military  personnel  and  training  planners  face  an  increasingly 
difficult  dilemma  on  how  to  stretch  their  manpower  to  meet  pressing  combat  unit  requirements 
and,  at  the  same  time,  to  deal  with  greater  training  loads  brought  about  by  new  weapons  systems 
and  high  personnel  turnover.  The  Services,  particularly  the  Army,  believe  they  need  to  reduce 
the  number  of  personnel  committed  to  the  training  function  in  order  to  free  skilled  and  experi¬ 
enced  key  NCOS  and  officers  for  field  duty. 

'This  paper  offers  analysis  suggestinq  on?  answer  to  this  serious  training  manpower  problem-- 
more  training  performed  by  contractors,  training  Lv  contractors  :s  certainly  not  new  but  it  has 
been  largely  utilized  in  the  past  for  limited,  h i ah i y  specialized,  often  one-time,  training 
efforts  associated  with  the  introduction  of  new  weapon  systems.  A  broader-scaled  contractor 
training  effort  might  offer  better  training  continuitv,  more  professional  staffing,  ways  to  meet 
capital  investment  costs  involving  new  sophisticated  computer-based  training  equipment,  and  yet 
produce  a  quality  trained  technician  in  a  cost,  effective  and  expeditious  manner.  A  pertinent, 
limited  case  study  provides  illumination  in  this  area.  The  paper  also  addresses  the  disadvant¬ 
ages  and  problems,  such  as  inflated  costs,  associated  with  a  contractor  developed  and  run  train- 
:ng  program  and  suggests  contractor  responsibilities  in  this  regard.  The  anticipated  manpower 
crisis  demands  a  search  for  solutions.  An  increased  traininq  role  for  contractors  might  ease 
the  armed  services  .Manpower  crunch  while  reducing  the  expected  industrial  competition  for  some 
of  the  same  manpower  resources.. 


INTRODUCTION 


The  demographic  predictions  about  forthcoming 
national  young  worker  personnel  shortages  are  by 
now  all  too  familiar.  The  problem  in  the  1990s 
whereby  one-third  of  all  18  year  olds  will  be  need¬ 
ed  by  the  military  and  whereby  the  mi  1 i tary/worker 
recruiting  pool  would  be  20  less  than  the  1978 
peak  has  been  aired  in  numerr  s  Armed  Services 
meetings  and  industry  conferences,  particularly, 
those  relating  to  personnel  and  training.  A  year 
ago  last  May,  the  problem  was  an  important  item  of 
discussion  at  San  D'?go  during  the  first  Annual 
Conference  on  Personnel  Traininq  Factors  in  Systems 
Effectiveness.  At  last  year's  A'  annual  confer¬ 
ence,  several  speakers  addressed  tne  subject.  The 
discussion  has  been  so  extensive  that  one  of  the 
1981  papers,  by  P.D.  Maher  of  Hughes  Aircraft 
Company,  had  as  its  title,  "Military  Personnel 
Shortages  Through  the  Year  2000  -  Fnough  Talk.' 

Let ’ s  Do  Something.'  1 .' " 

As  usual,  "doing  something"  is  considerably 
more  difficult  than  reviewing  and  dissecting  the 
problem.  Nevertheless,  a  number  of  very  positive 
proposals  have  been  advanced.  These  include  great¬ 
er  utilization  of  technology,  such  as  simulators, 
interactive  computer  based  training  devices,  inter¬ 
active  video  disc  systems,  arcade  qames ,  robotics, 
and  personnel  electronic  aids  to  maintenance.  Such 


technology  potentially  could  reduce  system  manpower 
req  ,i . ements ,  decrease  training  times,  conserve  re¬ 
sources,  and  trim  personnel  .upportinq  training. 

Other  ideas  on  meeting  tne  manpower  crunch 
have  included  the  expansion  ~  broaueninq  of  the 
workforce  by  qreater  use  of  women,  postponed  re¬ 
tirements,  more  extensive  use  of  less  capable 
people  and,  in  the  case  of  the  military,  the  use  of 
the  draft. 

Still  other  proposals  ha V'  called  for  enhanced 
worker/serviceman  productivity  (particularly  during 
the  early  years),  improved  selection  and  retention 
of  personnel,  ar.d  better  engineering  design  in 
weapons  systems  to  reduce  or  eliminate  manpower 
requirements.  At  the  sanie  time,  the  weapon  system 
designers  have  been  urged  to  i ncoroorate  extensive 
human  factors  front  end  analysis  with  respect  to 
trainabi 1 i ty ,  operability,  and  maintainability. 

Many  educators,  educational  psychologists,  and 
researchers  have  called  for  expanded  research  in 
the  area  of  training  transfer  in  order  to  derive 
the  most  education/training  efficiency  possible  in 
the  shortest  time  and  for  the  lyast  money.  The 
goal  would  be  to  keep  the  trainee  in  the  traini  ;g 
pipeline  as  short  as  possible  and  to  make  him  more 
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productive  once  on  the  job. 

All  these  ideas  have  merit  and  need  to  be 
pursued.  Obviously,  there  will  not  be  one  answer 
but  a  combination  of  many  ideas  that  will  be  needed 
to  meet  the  manpower  and  training  requirement  of 
the  1930s  and  1990s. 

ANOTHER  ANSWER 

With  the  thought  that  a  multiplicity  of 
answers  will  be  necessary,  yet  another  idea  is 
offered  toward  easing  the  forthcoming  crisis--a 
greatly  increased  training  effort  by  contractors. 
Training  by  contractors  is  certainly  not  new.  Many 
weapon  system  manufacturers  by  necessity  must  pro¬ 
vide  operability  and  maintainability  training  on 
their  products.  After  all,  they  know  tue  equipment 
and  can  train  the  technicians  because  they  design¬ 
ed  and  built  it.  In  addition,  companies  special¬ 
izing  in  training  services  and  training  equipment 
have  appeared  throughout  the  country.  These  train¬ 
ing  businesses  have  garnered  training  contracts 
from  both  industry  and  the  military.  Primarily, 
what  is  being  suggested  here,  however,  is  that  the 
scope  of  the  current  effort  be  notably  increased 
and  that  the  military  consider  contracting  out 
their  training  effort  in  areas  not  previously 
thought  appropriate  for  non-military  hands. 

Several  string  and  cogent  arguments  can  be 
made  for  this  proposal.  First,  a  stronger  and  more 
comprehensive  contractor  training  effort  would  free 
key,  critically  needed,  experienced,  highly  skilled 
non-commissioned  officers  and  officers  for  field 
duty.  These  personnel  are  usually  the  "cream-of- 
the-crop"  in  the  Armed  Services  and  placing  them 
be'k  into  field  leade. ship  positions  greatly  en¬ 
hances  military  readiness.  The  services  havp 
actively  sought  ways  to  reduce  their  military 
training  starfs,  partially  to  reduce  training  costs 
but  mainly  for  the  aforementioned  reason.  Recently, 
the  U.S.  r'r  Force  has  strenuously  complained  about 
the  relative  high  proportion  of  inexperienced  air¬ 
men  in  complex  specialties. 

Second,  better  training  continuity  will  re¬ 
sult  with  a  more  .table  contractor  staff.  The 
Armed  Forces  with  their  rotation  of  personnel  to 
overseas  positions,  and  ..ver:  with  stateside  assign¬ 
ments,  inherently  have  a  continuity  problem.  Con¬ 
siderable  eftort  must  be  expended  to  prepare  mili¬ 
tary  instructors  for  their  trainina  responsibilities. 
Often  the  new  military  instructor  must  be  sent  to 
special  contractor  offered  courses  (sometimes 
called  Type  I  training)  on  new  equipment,  Thus, 
maintaining  continuity  is  a  custly  and  time-consum- 
ir.g  problem  for  the  military. 

Third,  a  contractor  training  effort  would  pro¬ 
vide  a  high  technology,  state-of-the-art  approach 
to  training  without  the  large  capital  investment, 
in  similar  programs  being  required  by  the  Armed 
Services.  Many  companies  and  corporations  have 
heavily  invested  in  sophisticated  interactive 
computer  based  video  disc  training  systems  both  fo- 
commencial  selling  of  training  and  to  strengthen 
their  own  internal  ‘raining  programs.  The  31  May 
1982  issue  of  Aviation  Week  provided  an  excellent 
example  in  its  article  concerning  Boeing  Company* 
approach  to  flight  and  maintenance  training  for  its 
new  generation  of  t  ansports.  The  thrust  of  the 
article  was  Boeing's  fforts  to  use  new  technology 


(simulators  and  computers)  to  simplify  the  training 
of  pilots  and  mechanics.  Most  companies,  like 
Boeing,  would  welcome  greater  use  of  their  expen¬ 
sive  instructional  systems  at  company  training 
centers.  The  military  could  contract  with  a 
commercial  firm-  utilizing  their  advanced  computer 
based,  video  Jicc  equipped  learning  center's  to 
accomplish  specific  training  tasks.  Contract 
train. ny,  however,  should  not  be  viewed  as  simply 
sendinn  the  military  t^inee  to  a  company.  There 
may  be  many  times  when  travel  costs  or  other  eco¬ 
nomic  factors  would  dictate  having  the  contractor 
administer  training  on  a  military  instal 1ation. 
During  the  NSIA  spnsored  Army  Training  Technology 
Conference  at  Williamsburg,  Virginia,  in  Fef pary 
of  this  year,  a  tour  of  the  Army's  helicopter 
maintenance  training  at  Ft  Eustis  revealed  several 
prime  candidates  for  a  contractor  training  effort. 

Going  one  step  further,  taking  training  to  the 
trainee  in  the  field  could,  at  times,  best  be  per¬ 
formed  by  a  contractor.  Lt  Gen  Julius  Becton  of 
TRADOC  has  called  Tor  more  study  of  reducing  school 
time  by  field  training.  Dr.  Davis  S.  C.  Chu, 
Director,  Plans,  Analysis  and  Evaluation  for  the 
Secretary  of  Defense,  has  talked  about  contract 
support  (which  could  include  training)  on  bov-rd 
aircraft  carriers,  even  in  forward  areas. 

Obviously,  such  contractor  training  would  occur 
most  freauently  where  small  numbers  of  high  skill 
technicians  need  training  on  a  sophisticated  piece 
of  equipment.  It  is  here  that  a  contractor  can 
make  his  most  valuable  contribution  while  holding 
down  time  in  the  training  pipeline. 

Still  another  argument  for  contract  training 
is  the  more  professional  staffing  that  is  possible. 
This  is  not  to  say  that  there  are  uot  quality 
military  and  civil  service  instructors,  there  are. 
However,  the  turnover  problem  often  reduces  the 
concept  of  training  professionalism.  Currently, 
in  the  ICBM  force,  as  ar.  example,  training  at 
the  operating  command  level  is  being  accomplished 
by  technicians  selected  from  the  work  force  and 
designated  "instructors."  The  fallacy  of  this 
procedure  is  (1)  that  manning  documents  are  not 
structured  to  provide  additional  manning  for 
"instructors,"  (2)  the  work  force  is  effectively 
reduced  and  (3)  although  ohe  "instructors"  may  be 
exceptional  technicians,  their  auilities  as  quali¬ 
fied  instructors  may  be  seriously  lacking  and  they 
may.  in  fact,  be  adding  to  the  unit's  training 
burden.  In  the  military  schools,  the  instructor 
is  typically  identified  as  a  skilled  technician 
first  and  foremost  and  only  secondarily  assumes 
the  role  as  a  trainer/educator.  He  frequently  has 
only  one  stint  as  an  instructor.  In  contrast,  a 
business  involved  in  providing  training  services 
recruits  and  hires  an  experienced  instructor  who 
is  expected  to  continuously  exhibit  the  trad. marks 
of  a  competent  trainin,  professional. 

Turning  more  to  contract  training  during  the 
manpower  crunch  period  would  also  ease  the 
inductrial/military  competition  for  the  same 
individuals  in  the  dwindling  pool  of  technicians 
and  training  professionals.  The  numbers  might 
not  be  large  but,  because  of  the  skills  involved, 
they  would  be  significant.  Any  stretching  of  the 
manpower  resource  will  bo  important. 

Unfortunately ,  as  with  most  proposals  of  this 
nature,  there  are  disadvantages  that  must  be 
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weighed.  Most  formidable,  at  least  at  first 
glance,  is  the  cost  that  contractor  training 
would  entail.  Because  training  would  be  openly 
purchased,  costs  would  become  supremely  visible, 
like  other  parts  of  a  weapon  system.  Some  train¬ 
ing  associated  with  the  introduction  of  a  sophis¬ 
ticated  new  equipment,  say  a  killer  satellite, 
would  be  expensive  indeed.  A  word  of  caution, 
however,  Dr.  David  Chu  points  cut  that  the  cost 
of  training  conducted  by  the  military  d^es  not 
ah. ays  reflect  indirect  costs  cuch  as  personnel 
retirement  and  inflationary  (COL)  increments. 

Another  disadvantage,  associated  with  most 
procurement  actions,  is  the  loss  of  some  flexi¬ 
bility.  Changes  in  training  programs  could 
become  more  difficult  because  of  contractual 
rigidities.  Planning  and  requesting  training 
would  assume  greater  significance  because  time 
would  have  to  be  allowed  for  procurement  pro¬ 
cedures. 

In  addition,  some  will  argu°  that  the  con¬ 
tractors  cannot  respond  to  wartime  expansion 
requirements  or,  perhaps,  not  effectively  train 
for  war  situations. 

There  may  be  other  disadvantages  but,  like 
thu  aforementioned,  they  are  not  likely  to 
counterbalance  the  gains  offered  by  increased 
contractor  training. 

A  CASE  IN  POINT 

How  might  more  contractor  training  work?  Let 
us  take  the  new  M-X  ICBM  as  an  example. 

Currently,  b,  contract  considerable  training 
planning  is  underway  by  various  corporations 
working  on  this  new  missile.  This  large  four 
stage  ICBM  incorporates  new  technology  requiring 
extensive  training  for  the  training  of  ISP 
developers;  instructors  for  Air  Trailing  tonmand. 
Strategic  Air  Com  .and,  and  Air  Force  Logistics 
Command;  Missile  Test  personnel;  and  the  initial 
cadre  for  the  operational  base  of  the  deployed 
missile.  The  military  ISO  developers,  instructors, 
and  initial  cadre  personnel  are  expected  to  plan, 
to  assemble,  and  to  execute  a  training  program 
to  satisfy  the  manning  needs  for  the  new  weapon 
system  by  1986.  This  is  essential  and  good  but 
it  doesn't  go  far  enougi. 

What  makes  greater  sense,  particularly  in 
light  of  th<-  anticipated  shortage  of  skilled 
technicians,  is  for  the  contractors  to  carry  their 
training  planning  into  action.  When  the  Air  Force 
lets  the  weapon  system  contract  or  contracts ,  a 
complete  training  package  should  be  included.  Not 
onTy  would  contractors  plan  out  they  would  complete 
the  ISD  process  and  then  conduct  the  training. 

The  contractors,  l'ke  Martin  Marietta  in  M-X,  have 
by  necessity  organized  and  administered  a  training 
program  for  their  own  personnel.  The  training 
staffs  and  courses  have  been  assembled  and  train¬ 
ing  initiated.  Instead  of  training  Air  Force 
personnel  to  train  ether  Air  Force  personnel,  the 
contractor  would  then  build  o.i  the  contractor 
training  already  underway  and  assume  the  whole 
training  burden.  In  addition,  the  current,  System 
Reg ui remen ts  Analysis  and  Logistics  Support 
Analysis  contribute  greatly  to  the  ISD  effort  in 
effect  negating  the  need  for  the  long,  costly  !:SD 
development  by  the  mil.tary.  In  the  case  of  M-X, 


the  contractors  would  provide  the  training  for  the 
initial  operational  base  personnel  and  any  neces¬ 
sary  tollow-on  training  to  support  the  operational 
missile.  In  other  words,  tne  contractor  or 
contractors  would  provide  complete  educational 
training  services  thereby  freeing  military  men  and 
women  for  other  duties. 

In  brief,  what  is  being  suggested  is  that  the 
Armed  Services  let  the  contractors  build  their 
training  program  as  they  also  build  the  hardware. 

IMPORTANT  CONSIDERATIONS 

Whether  the  contractor  training  effort  is 
part  of  the  development  and  deployment  of  new 
weapon  systems  or  the  contracting  out  of  an  on¬ 
going  military  training  program,  certain  elements 
must  be  stressed  to  insure  success. 

First,  contractors  must  deliver  and  maintain 
profesr ional ,  guality  training.  As  with  any  pur¬ 
chase  agreement,  the  customer,  in  this  case  the 
nation  and  the  Armed  Services,  has  a  right  to 
expect  that,  after  training,  defense  personnel 
will  be  able  to  perforin  their  ever  increasingly 
complex  tasks.  This  means  the  contractors  must 
institute  rigorous  quality  controls  rver  training 
just  as  expected  on  hardware  or  software.  No 
slipshod  training  can  be  tolerated.  Because  of 
the  nature  of  the  educational  process  arnd  the 
sensitivities  of  educators  to  the  imposition  of 
quality  standards,  this  is  not  as  easy  as  it 
sounds. 

Second,  contractors  must  implement  stringent 
cost  disciplines  in  their  training  programs.  As 
previously  noted,  businesses  involved  in  training 
can  often  take  greater  advantage  of  new  technology 
to  strengthen,  speed,  and  enhance  the  efficiency 
of  their  training.  At  the  same  time,  restraints 
must  he  exercised  in  continual  procuring  of  newest 
state-of-the-art  training  equipment,  thereby 
constantly  escalating  costs.  Similar  to  design¬ 
ing,  the  temptation  of  over- engineering  must  be 
avoided.  To  make  it  possible  for  the  Defense 
Department  to  contract  for  training,  costs  must  be 
kept  re.  istic  and  with  appropriate  justification. 
Also,  contractors  need  to  be  prepared  to  offer 
"lire  cycle"  costs  in  their  training  proposals. 

On  the  other  side  of  the  coin,  the  Defense 
Department  should  fully  consider  and  weigh  all 
costs  associated  with  military  manpower,  including 
such  elements  as  retireme-.it  and  inflationary  pay 
increases.  This  becomes  important  when  assess¬ 
ing  or  comparing  the  economics  of  training 
conducted  by  the  military  with  training  conducted 
by  industry. 

A  fourth  consideration  is  the  matter  of 
responsiveness  to  tracing  requirements.  Not  only 
mast  contractors  plan  and  prepare  to  offer  train¬ 
ing  packages,  along  with  their  weapon  systems, 
they  need  a  receptivity  and  capability  to  adjust 
to  changes  in  on-going  training  programs.  One 
of  the  major  problems  in  military  training  has 
been  the  many  fluctuations  ir  the  flow  of  person¬ 
nel  through  the  training  pipeline.  It  should  be 
kept  in  mind  that  training  people  will  forever  be 
a  dynamic  enterprise  and  contractors  involved  in 
tiis  enterprise  must  recognize  this  and  maintain 
a  greater  flexibility  than  normally  considered 
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with  contractual  specifications. 

A  fifth  factor  needing  consideration  is  the 
outlook  of  Defense  personnel  toward  contracting 
training.  Defense  officials  will  need  to  be  more 
open  to  proposals  in  this  regard,  includinq  train¬ 
ing  exclusively  and  traditionally  thought  of  as 
administered  only  by  military  units.  In  some 
caseb,  traditional  military  training  programs 
could  be  broken  down  into  parts,  some  ot  which, 
especially  combative  or  security  sensitive,  would 
remain  militarily  conducted,  v.hile  others  would  be 
handled  by  commercial  firms.  In  the  last  few 
years,  there  has  been  a  definite  gain  in  the 
receptiveness  of  high  officials  but  this  must  be 
extended  to  lower  levels  in  order  to  make  a  bigger 
impact  on  the  manpower  problem. 

Sixth,  serious  thought  must  be  given  to  more 
automated  tutorial  type  training  for  recurring 
and  long-range  programs.  The  military  services 
commit  important  resources  to  administering  train¬ 
ing  that  is  required  on  an  annua:  basis.  Con¬ 
tractors  could  step  into  this  picture  with  a 
valuable  service.  Because  of  the  often  low 
priority  of  these  training  programs  and  the  need 
for  the  most  economical  training  possible, 
commercial  firms  would  need  to  rely  on  technology 
intensive  (individualized  interactive  video  disc) 
versus  labor  intensive  approaches  to  this  training 
a^ea.  In  concept,  a  contractor  run  learning  center 
would  offer  a  self-paced  traininy  package  to  the 
trai  ,oo  who  progresses  with  little  supervision  to 
a  test  and  final  fulfillment  of  the  training  re- 
qui rernent. 

Finally,  contractors,  just  as  their  military 
counterpa. ts ,  must  continually  tie  training  to 
mission  effectiveness.  Since  we  are  talking 
basically  about  military  training,  the  question 
should  forever  be  raised,  "What  happens  in  a  war¬ 
time  situation?"  Is  tue  training  pertinent  to 
war  needs?  Can  readiness  be  maintained  and  can  a 
fighting  effort  be  sustained?  What  about  surge 
training  capacity  to  support  larger  war  forces? 

Some  individuals  have  called  attention  to  this 
point,  like  Dr.  Arthur  Siegel  of  Applied  Psycho¬ 
logical  Services  Corporation,  who  takes  a  dim  view 
of  all  the  talk  about  simulation  behavioral  model¬ 
ing  that  cannot  reflect  wartime  requirements. 
Likewise,  all  contractors  must  look  critically  at 
their  training  in  the  same  regard.  To  paraphrase 
an  oft  quoted  remark,  "We  don't  want  to  build  the 
best  training  system  that  can  be  built  ar.d  then 
have  a  war  come  along  and  ruin  the  whole  thing." 


The  foregoing  discussion  suggests  yet  another 
way  to  respond  to  the  nation's  forthcoming  man¬ 
power  crisis.  It  is  clear  a  multifaceted  solution 
to  this  difficult  problem  is  needed  and  tra.ninq 
by  contractors  can  be  part  of  the  overall,  work¬ 
able  response.  It  does  offer  a  number  of  signifi¬ 
cant  advantages.  At  the  same  time,  its  effective¬ 
ness  could  depend  on  a  variety  of  factors.  With 
their  careful  consideration,  contractor  training 
could  nelp  promote  teamwork,  instead  of  deadly 
competition,  between  government ,  the  military, 
and  industry  in  stretching  limited  people  resou-ces. 
A  far  broader  scaled  contractor  training  effort 
could  this  b-  a  significant  contributor  to  a 
solution  of  a  national  problem  while  producing 


quality  trained  technicians  or  operators  in  a  cost 
effective  and  expeditious  manner. 
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ABSTRACT 

^Effective  communications  between  Government  and  Industry,  so  necessary  for  a  successful  pro¬ 
curement  effort,  have  often  been  missing  in  the  past.  This  has  resulted  in  cost  overruns,  unreal¬ 
istic  approaches  and  expectations,  missed  schedules,  and  finally  unuseable  and  inadequate  deliver¬ 
ables.  Ir,  this,  the  Age  of  Atari,  the  training  community  has  used  the  game  approach  to  success¬ 
fully  train  personnel  in  many  areas.  This  paper  uses  that  same  approach  to  reach  both  Government 
and  Industry  in  an  effort  :o  remedy  the  communication  situation.  The  game  is  based  on  some  of  the 
milestones  which  occur  during  a  procurement.  The  rules  are  the  policies  and  guidelines  which  both 
players  must  consider  to  progress  toward  the  game's  end.  Both  Government  and  Industry  win  through 
the  exchange  of  information,  increased  communication,  and  team  effort.  In  conclusion,  it  was  found 
that  effective  communications  are  an  essential  Key  to  success  in  the  game.*  . — 


SECTION  i 
INTROOUC I  I  ON 


The  procurement  cycle,  as  both  Government  and 
Industry  know  it  today,  is  often  full  of  misun¬ 
derstanding,  finger  pointing,  and  adversary  re¬ 
lationships.  Each  side  in  this  conflict  tradi¬ 
tionally  spends  far  too  much  time  in  trying  to 
determine  who  is  right,  ratner  than  moving  for¬ 
ward  in  a  joint,  mutually  beneficial  effort  to 
resolve  procurement  problems.  Effective  co'iinu- 
nication  from  both  sides  which  could  increase  the 
likelihood  of  successful  contract  completion  is 
of  ten  missing. 

The  purpose  of  this  paper  is  to  aid  both  the 
Government  and  Industry  in  achieving  mere  effec¬ 
tive  communication  in  this  area,  specifically 
during  the  procurement  cycle.  The  approach  taken 
has  been  to  present  the  problem  in  a  game  format. 

The  name  of  this  game  is  PC  (Procurement  Comnu- 
nication).  Everyone  can  participate  in  this  type 
of  game.  There  are  no  limits  on  the  number  of 
players,  but  they  are  generally  grouped  into  two 
teams,  Industty  and  Government.  The  goal  of  the 
Government  team  is  "to  obtair  a  good  product  for 
a  fair  price,"  while  the  gc  i  of  the  Industry 
team  is  "to  obtain  a  reason ?b • e  profit." 

The  game  setting  is  the  Acquisition  arena.  The 
object  of  the  ga^e  is  for  a  team  to  successfully 
progress  through  five  procure-  -nt  milestones: 

Requirements  Definition 
Request  for  Proposal 
Contract  Award 
Post  Award 

Modification,  Completion  or  Termination  of 
Contract 


Each  milestone  has  its  own  peculiarities  and  as¬ 
sociated  policies  which  will  present  a  communi¬ 
cations  challenge  to  all  the  participants.  The 
risks  for  each  team  increase  as  they  progress 
through  the  milestones. 


The  game  consists  of  five  innings.  The  Govern¬ 
ment  is  the  Visiting  team.  Industry  is  the  home 
team.  Each  team  will  have  an  "up"  in  every  in¬ 
ning. 

To  keep  the  game  interesting,  there  are  a  few 
constrai nts  'to  be  considered.  On  the  Government 
side  of  the  board,  one  constraint  was  aptly  de¬ 
scribed  in  a  theorem  by  B.  Schemer,  Editor  of 
the  Armed  Forces  Journal:  < 

"When  faced  with  a  20-year  threat,  Govern¬ 
ment  responds  with  a  1 5-yenr  program  in  a 
Five-Year  Defense  Plan,  managed  by  3-year 
personnel,  funded  with  single  year  appro¬ 
priations." 

On  the  opposite  side  of  the  b^ard,  Industry  often 
has  high  mobility  of  personnel  during  a  procure¬ 
ment  cycle.  Those  that  understand  what  is  re¬ 
quired  and  desired  ’* n  the  beginning  are  not 
necessarily  those  who  execute  the  contract. 

Communication  difficulties  between  Industry  and 
Government  merely  add  to  any  existent  problem. 
Through  u:e  of  the  PC  Game  approach  and  effective 
communications,  Government  and  Industry  can  both 
become  "smart  players"  and  end  up  a  winning  team. 

MILESTONE  #  1 
REQUIREMENTS  DEFINITION 

The  Government  is  continually  considering  new 
requirements  for  procurement.  Mastery  of  Mile¬ 
’S  tone  #1  requires  that  Industry  be  advised  of 
these  requirements.  The  methods  by  which  the 
Government  makes  its  requirements  known  to  In¬ 
dustry  are  varied.  One  approach  is  the  annual 
I nterservi ce/ Industry  Training  Equipment  Con¬ 
ference,  which  presents  a  perfect  forum  for  ad- 
vising  Industry  of  Government's  requirements. 
Last  year’s  conference  was  attended  by  668  In¬ 
dustry  and  461  Government  representatives. 


The  Government  strives  ta-4mprove  its  procurement 
practices  through  various  methods  of  communica¬ 
tion.  An  excellent  example  of  this  was  a  series 
of  three  conferences  (Atlanta  I,  II  and  III)  held 

BEST  AVAILABLE  COPY 


!' 

. 


447 


by  the  U.S.  Army  m^ter'a!  acquisition  community 
in  May  1974,  February  1975,  and  May  1976.  In¬ 
dustry  representat i ves  met  with  Army  representa¬ 
tives  to  review  an  Acquisition  Process  that  was 
beset  with  many  probleiis.  Communicat  ioris  flowed 
freeiy  and  this  was  quite  evident  by  the  tone  of 
Major  General  George  Sammet,  Jr.'s  comments: 

"Late  in  1973,  the  Army's  Acquisition  Pro¬ 
cess  ,«as  an  object  of  increasing  criticisms. 
The  process  was  viewed  as  cumbersome, 
noriresponsi  ve,  and  layered  with  bureaucracy 
that  either  couldn't  or  wouldn't  perform. 
Criticisms  caine  from  every  quarter. 

We  didn't  like  what  we  heard  and  didn't  be¬ 
lieve  it.  No  manager  likes  criticism. 

Nevertheless,  the  perception  that  we  were 
not  effective  and  were  hard  to  do  business 
with  remained.  Whether  this  perception  w  *, 
indicative  of  the  true  state  of  our  acqusi- 
tion  capabilities  or  not.  something  had  to 
be  done.  We  did  suspect  that  the  outside 
world  saw  all  of  our  weaknesses,  and  little 
of  our  strengths. 

We  decided  the  best  course  of  action  was  to 
get  this  probien  out  in  the  open  and  meet 
directly  with  most  of  our  severest  critics  - 
our  industrial  suppliers." 

Twenty-one  acquisitions/i  riitiatives  were  estab¬ 
lished  at  these  meetings.  One  of  these  initia¬ 
tives,  Acquisition  Initiative  No.  6,  "Advance 
Research  and  Development  (R&D)  Information," 
recommenoed  release  of  planning  summaries  (less 
funding  data)  and  the  conuuct  of  advanced  brief¬ 
ings.  Based  on  the  results  of  these  three  con¬ 
ferences,  the  Project  Manager  for  Training 
Devices  (PM  TRADE)  established  a  policy  whereby 
their  requirements  documents  would  be  released 
to  Industry  on  request  and  briefings  would  be 
conducted  informally  within  reasonable  limits 
considering  workload,  availability  of  personnel, 
and  factors  of  that  nature. 

However,  before  this  information  can  be  furnished 
to  Industry,  the  Government  must  have  accurately 
defined  their  ,  equ  ireiiieiits.  Requirements  should 
fulfill  user  desires  and  at  the  same  time  be 
technically  feasible  and  ai  fordable. 

Meanwhile,  before  Industry  can  master  the  Re¬ 
quirements  Definition  Milestone,  they  must  es¬ 
tablish  their  corporate  performance  objectives 
and  policy  guidelines,  as  well  as  recognize  wiiere 
their  capabilities  lie.  Performance  objectives 
include  U  .  determination  of  the  type  of  market, 
possible  market  share,  anticipated  dol  iar  amount, 
and  return  on  investment  (KOI)  desired.  Policy 
guidelines  re  1  at  i  v  _  to  Quality  assura»ice,  pric¬ 
ing,  public  relations,  advertising  and  financing 
must  be  decided. 

Once  these  objectives  r.nd  policies  are  estab¬ 
lished  and  corporate  capabilities  are  recognized. 
Industry  must  then  obtain  information  about  t tie 
present  and  future  characteristic*  of  the  Gov¬ 
ernment  market.  At  the  same  time,  they  must 
further  their  own  capabilities  through  indepen¬ 
dent  research  and  development,  updating  of  resi¬ 
dent  skills  and/or  experience  through  other  ccn- 
trac  ts. 


Milestone  achievement  is  now  measured.  In  the 
top  half  uf  the  first  inning,  the  Government,  is 
asked  the  following  questions: 

1)  Have  t.' e  requirements  (not  just  a  wish 
list)  been  accurately  defined? 

2)  Has  Industry  been  informed  of  the  re¬ 
quirements? 

3)  Are  the  requirements  achievable  with 
state-of-ttie-art  technology? 

If  the  answer  is  "no”  to  any  of  the  three,  the 
side  must  retire.  To  continue  the  game  and  reach 
the  milestone,  the  requirements  must  be  rede¬ 
fined,  Industry  must  be  informed,  and  the  tech¬ 
nology  must  be  available.  The  game  continues 
when  all  answers  are  "yes". 

In  the  bottom  half  of  the  first  inning,  Industry 
is  asked  the  following  questions: 

1)  Have  performance  objectives  been  deter- 
mi ned? 

2)  Has  the  marketplace  been  identified? 

3)  Have  corporate  capabilities  been  ex¬ 
panded  and  updated? 

If  the  answer  is  "no"  to  any  of  the  three,  the 
side  is  retired.  To  continue,  corporate  per¬ 
formance  objectives  must  be  determined,  the  mar¬ 
ketplace  where  company  capabilities  most  fit  must 
be  identified,  and  capabilities  must  be  updated 
and  expanded.  The  game  continues  when  all  an¬ 
swers  are  "yes." 

Lack  of  communication  with  Government  as  to  ac¬ 
tual  require  :ents  may  cause  Industry  to  waste 
money  in  the  wrong  areas  and  elevate  overhead 
rates,  causing  them  to  no  longer  ue 
competitive.  INDUSTRY  LOSES. 

If  Government  fails  to  communicate  its  require¬ 
ments  to  Industry,  research  and  development  ef¬ 
forts  may  proceed  along  and  not  meet  the  needs 
of  anyone.  The  GCVERNMEM  LOSES. 

Effective  communication  can  turn  this  into  an 
"EVERYBODY  WINS"  situation  wnere  Industry  keeps 
Government  informed  on  the  state-of-the-art,  as 
well  as  new  research  breakthroughs.  In  return, 
the  Government  will  be  aole  to  plan  developments 
around  the  latest  technology  while  keeping 
industry  apprised  of  the  requirements. 

MI LES10NE  #  i 
REQUEST  FUR  PROPOSAL 

Thu  next  play  belongs  to  the  Government,  who 
Tortr ally  cormiunicutes  with  Industry  by  publishing 
a  synopsis  of  go  impending  procurement  in  the 
i owne re e  Business  Dai  ly  (LBU).  Tins  sues 

prospective  offerors  an  equal  opportun.iy  tc 
participate  in  the  acquisitions. 

Durinq  this  milestone.  Industry  must  obtain  as 
much  information  on  the  Government's  requirements 
as  possible.  This  includes  military  philosophy, 
weapons  or  system  requirements,  declining  or  ex¬ 
panding  markets,  and  general  "G-3".  Industry 
must  obtain  guidance  to  optimize  selection  o' 
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biddable  RFPs,  which  will  maximize  the  KOI  and 
utilize  capabilities  and  available  facilities. 
Sufficient  advanced  lead  time  in  relation  to  the 
RFP  can  only  be  obtained  through  gocd  communica¬ 
tions  with  the  Government  as  to  its  purpose  and 
requirements  as  defined  in  Milestone*  1.  This 
intelligence  can  improve  Industry's  competitive 
position.  Analy^s  of  the  Government  require¬ 
ments  can  aid  in  any  Bid  No  Bid  decision  that 
must  be  made  at  this  time. 

The  Request  for  Proposal  (RFP)  is  a  major  ac¬ 
quisition  link  that  must  be  carefully  prepared 
by  the  Government.  Important  documents  embedded 
in  the  RFP  are  the  Statement  of  Work  (SOW)  and 
the  Specification.  These  documents  should  be 
accurate  and  r or. -ambiguous,  since  they  define  the 
level  of  effort  required  of  the  offeror  to  ac¬ 
complish  the  task.  They  also  play  a  significant 
role  in  proposal  evaluation,  and  ultimately  be¬ 
come  the  standard  for  measuring  the  Contractor's 
perf  ormance 

Industry's  answer  to  the  RFP  requires  stweru! 
types  of  communir  at  ion.  Witnin  each  corporation, 
the  approach  must  be  c ’early  communicated  to  all 
proposal  writers.  Trie  requirements  must  be  un¬ 
derstood  before  they  can  be  answered  in  a  tech¬ 
nically  acceptable  nvnner.  Industry  proposal 
writers  must  inform  the  Government  of  the  offer¬ 
or's  intentions  in  a  manner  which  cannot  be  mis¬ 
understood  or  misread.  At  the  same  time,  they 
must  persuade  Government  that  theirs  is  THE  best 
approach.  Developed  "Should  Costs"  must  be 
accurate,  and  pricing  inputs  must  be  realistic. 

The  Government  must  avoid  providing  any  informa¬ 
tion  relative  to  proposed  or  contemplated  ac¬ 
quisitions  to  any  prospective  offerors  without 
providing  it  to  all,  since  it  might  give  them  an 
advantage  ever  other  offeror.  Not  playing  by 
these  rules  may  earn  the  Government  team  a  stiff 
penalty,  since  all  prospective  offerors  must  be 
provided  exactly  the  same  information. 

A  Pre-Proposal  Conference  is  sometimes  held  to 
assure  that  the  scope  of  work  is  understood  or 
to  clarify  portions  of  the  RFP.  The  questions 
and  answers  are  recorded  and  furnished  to  all  the 
offerors.  Requirement  weaknesses  or  ambiguities 
are  sometimes  revealed  at  this  stage,  and  a  for¬ 
mal  amendment  must  be  made  to  the  RFP.  Such 
changes  to  the  solicitation  package  must  be  tur- 
n  i  sheu  to  all  f  inns. 

The  Government  is  responsible  ♦or  safeguarding 
all  information  contained  in  an  Offeror's  pro¬ 
posal  This  includes  bids,  quotations,  descrip¬ 
tive  1  i  terature/inatena  1  or  special  technical 
data.  In  addition,  any  discussions  held  with 
offerers  about  their  proposals  must  not  disclose 
intorma* ion  relative  to  any  other  compel inq  of¬ 
feror's  proposal. 

‘nformzMon  provided  in  trie  Industry  response  to 
the  RFP  is  privileged  and  confidential.  When  it 
is  provided  to  the  Government  i i  qood  faith,  it 
is  assumed  that  Government  eyes  only  will  view 
it. 

lii  the  top  half  of  the  second  inninq,  the  Gov¬ 
ernment  must  answer  the  following  questions: 

1)  Is  the  RFP  accurate  and  non- ambiguous? 


2)  Has  the  same  information  been  furnished 
to  ali  prospective  offerors? 

3)  Has  all  contractor  confidential  informa¬ 
tion  been  safeguarded? 

Any  "no"  answers  cause  the  side  to  retire.  The 
milestone  can  be  achieved  by  clear  and  concise 
wording  of  the  RFP,  by  assuring  that  all  prospec¬ 
tive  contractors  receive  the  same  information, 
and  by  safeguarding  the  confidentiality  cf 
contractor-furnished  information. 

In  the  bottom  half  of  the  inning.  Industry  must 
answer  the  following  questions: 

1)  Has  the  bid  decision  been  decided  from 
a  thorough  analysis  of  available  intel- 
1  igence? 

2)  has  the  RFP  been  answered  in  a  manner 
which  cannot  bt  ’’sunderstood? 

3)  Have  realistic  co;ts  been  proposed? 

Any  "no"  answers  cause  the  side  to  retire.  The 
milestone  will  be  reached  by  using  all  informa¬ 
tion  available  to  make  the  decision  to  bid,  by 
understanding  the  RFP  and  answering  it  in  a  non¬ 
confusing  manner,  and  by  accurately  projecting 
the  price  of  the  project. 

An  offeror  in  developing  proposl  costs  must  be 
very  careful.  Proposals  that  are  too  high  in  ay 
be  non-competitive  and  proposals  that  are  too 
low  endanger  potential  profit.  Either  way,  the 
Of c  PR  OR  LOSES. 

Whenever  Government  fails  to  communicate  its 
needs  to  Industry,  the  results  are  an  item  of 
insufficient  quality  (performance)  or  a  higher 
than  anticipated  cost.  The  results  can  be 
equally  undesirable  if  Government  fails  to  un¬ 
derstand  the  proposal  from  Industry.  If  such  a 
proposal  is  accepted  under  a  cost  type  contract 
and  runs  significantly  higher  than  envisioned, 
added  costs  result.  The  total  liability  was  not 
understood,  and  GOVERNMENT  LOSES. 

both  GOVERNMEN!  and  the  Oi  lEROK  ChN  WIN,  through 
their  ability  to  effectively  comnunicate  via  the 
wri tten  word. 

MILESTONE  *  3 
CONTRACT  AWaRU 

Prior  to  contract  award,  an  in-depth  evaluation 
of  all  proposal's  received  must  be  performed  by  a 
Government  team  of  experts.  1  fie  evaluation  must 
be  thorough,  unbiased,  and  in  accordance  «itu  the 
factors  provided  by  the  RIP.  Industry  must  have 
confidence  that  the  Government  will  follow  the 
stated  plan. 

Of '  erors  must  sometimes  provide  c  larif  icat  lori 
regarding  specific  areas  of  v  proposal.  jum- 
pioposals  art;  found  tu  be  unacceptable  wfien  they 
fail  to  address  the  salient  pulnts  Of  the  RFP. 
Following  clan  tying  discussions,  offerors  may 
be  asxed  fur  their  “best  and  final  offer". 

Following  evaluation  by  the  Government,  the  suc¬ 
cessful  offeror  is  notified  and  a  formal  not  lo¬ 
ot  contract  award  is  published  in  the  Commerc e 
business  Dai ly. 


449 


rhf-  next  major  communication  area  is  the  negoti¬ 
ations  which  are  conducted  in  accordance  with 
s-rict  procedures.  The  purpose  of  negotiations 
is  to  allow  complete  understanding  regarding  what 
tnc  offeror  proposes  and  whether  it  meets  the 
Government's  requirement s,  and  finally  to  allow 
negotiation  on  the  cost  of  the  effort.  The  lat¬ 
ter  is  of  paramount  importance,  since  the  Gov¬ 
ernment  must  be  aware  of  its  liability  and  the 
Contractor  is  concerned  with  a  profit. 

During  inning  number  three,  the  Government  must 
successfully,  answer  the  following: 

1)  Has  the  best  contractor  (who  will  supply 
best  value)  been  chosen? 

Obviously,  the  wrong  choice  here  will  result  in 
a  less  than  optimal  procurement  effort. 

In  their  half  of  the  inning,  the  winning  Con¬ 
tractor  must  successfully  answer  the  following: 

1)  Will  we  assure  that  the  contract  award 
was  wel 1  deserved? 

When  both  answers  are  "yes",  the  game  progresses. 
The  Government  has  successfully  described  its 
needs,  and  the  Contractor  has  successfully  de¬ 
scribed  a  method  for  meeting  those  needs. 


MILESTONE  #  4 
POST  AWARD 

A  Post  Award  Conference  should  be  held  at  the 
Contractor's  facility  during  the  first  month  af¬ 
ter  contract  award.  Such  a  meeting  can  establish 
rapport  between  the  Government  and  the  Contrac¬ 
tor.  It  sets  the  tone  for  an  effective  working 
relationship,  advises  the  Contractor  of  the 
reporting  requirements,  and  clarifies  contractual 
poi nts . 

Informal  communications  can  now  proceed  on  a 
day-to-day  basis  at  the  working  level.  Reports, 
records,  amendments  to  the  scope  of  work,  or 
delivery  schedules  are  conveyed  on  a  formal  basis 
via  the  Government  contracting  officer.  Any 
amendments  related  to  deliverables  or  the  scope 
of  work  can  only  be  approved  by  the  contracting 
officer.  However,  the  contract  also  provides  for 
various  meetings  between  the  Contractor  and  Gov¬ 
ernment  relative  to  particular  program  aspects 
which  allow  interactive  communication. 

Both  Contractor  and  Government  management  must 
be  effective  communicators  within  their  internal 
environments.  Effective  internal  coiTinunications 
can  lessen  the  possibility  of  external  conflict 
when  each  team  interfaces  with  the  other. 

By  the  time  Post  Award,  the  fourth  milestone,  is 
in  sight,  the  Contractor  should  have  his  task  or 
job  clearly  defined.  The  requirements  of  the  RFP 
should  be  reduced  to  precise  and  detailed  work 
statements.  The  Specification  and  any  instruc¬ 
tions  necessary  to  undertake  and  complete  the  job 
should  be  understood. 

It  is  now  the  privilege  of  the  contractor  to  do 
the  job  as  he  sees  fit  within  the  limits  of  the 
contract.  The  tendency  to  ask  for  direction 
continually  from  the  Government  should  be  guarded 
against.  Good  internal  communications  must  be 
established.  Management  should  look  to  the 


written  terms  of  the  contract  for  guidance  and 
to  the  Government  contracts  admi  rii  strator  for 
i nterpretat ion. 

At  the  same  time,  the  Government  should  be  in  a 
surveillance  mode,  not  a  supervisory  one.  It 
should  not  dictate  how  the  Contractor  is  to  per¬ 
form,  but  rather  assist  the  Contractor  when 
necessary  to  ensure  that,  contrac tual ly ,  the 
Government  receives  what  it  bargained  for. 

Probably  the  single  most  important  factor  during 
Post  Award  is  establishing  informal  comnunication 
between  the  Government  and  the  Contractor  at  the 
working  level.  Initially,  the  Government  initi¬ 
ates  much  of  this  communication  to  assure  that 
the  Contractor  fully  understands  what  is  required 
under  the  contract.  However,  once  work  com¬ 
mences,  a  flow  of  progress  reports  and  problem 
identification  and  resolution  is  communicated 
back  to  the  Government  from  the  Contractor.  Such 
an  information  exchange  is  required  for  smooth 
progression  of  the  effort. 

During  their  respective  turns  in  inning  number 
four,  the  Government  and  the  Contractor  must  re¬ 
ply  in  the  affirmative  to  the  following: 

1)  Have  we  established  effective  communi¬ 
cation  methods  with  our  Contractor/the 
Government? 

2)  Are  we  allowing  our  Contractor/the  Gov¬ 
ernment  to  do  the  job  we  both  agreed  to 
during  contract  negotiations? 

If  there  are  minimal  problems  and  there  is  ready 
resolution  for  existent  ones,  and  effective  com¬ 
munication  methods  have  been  established,  the  two 
teams  can  successfully  advance  to  the  next  mile¬ 
stone. 

MILESTONE  #5 

MODIFICATION,  COMPLETION  OR  TERMINATION 

The  fifth  inning  and  final  milestone  are  a  cul¬ 
mination  of  all  the  pluses  and  minuses  which  have 
accumulated  throughout  the  game  for  each  team. 
The  milestone  can  take  three  separate  and  dis¬ 
tinctive  forms,  Termination  of  the  Contract, 
Modification  of  the  Contract,  or  Completion  of 
the  Contract.  It  is  during  this  period  that  each 
team  can  judge  the  effectiveness  of  their  com¬ 
munications;  and  retire  from  the  game  arena  to 
lick  their  wounds,  regroup,  or  successfully  com¬ 
plete  the  procurement. 

A  variety  of  reports,  l.e.,  progress  or  status, 
expenditures,  achievement  of  specific  technical 
directives,  etc.,  constitute  a  formal  record  of 
the  contractor's  performance.  These  reports 
provide  details  of  information  already  known  to 
both  parties.  However,  future  actions  of  both 
the  contractor  and  other  Government  agencies  are 
based  upon  these  reports. - 

The  item  developed  under  a  contract  must  be 
tested  in  accordance  with  a  formal  test  plan.  A 
great  deal  of  two-way  formal  and  Informal  com¬ 
munications  are  involved  in  the  process.  When 
the  item  is  successfully  tested,  the  result  is 
contractual  progress.  When  the  test  Is  not  suc¬ 
cessful,  sometimes  a  modification  to  the  contract 
Is  necessary,  There  are  a  few  contracts  that  are 
not  successfully  completed  but  are  terminated  fer 
various  reasons. 
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Termination  or  the  contract  may  be  the  disas¬ 
trous  result  of  a  complete  failure  to  communi¬ 
cate  effectively.  No  meeting  of  the  minds  has 
occurred,  no  real  understanding  of  the  problens 
has  happened,  and  each  participant  in  the  game 
must  retire  in  defeat. 

The  middle  ground  between  failure  and  success  in 
the  Procurement  Game  is  contract  modification. 
Tfi  rough  a  bi-eakdown  in  the  commur,  i  cat  ions  pro¬ 
cess,  the  contract  has  gotten  off  the  track.  The 
game  plan  is  then  to  repair  the  damage  by  re¬ 
establishing  the  communication  link  to  permit 
redefinition  of  the  requirements  and  rethinking 
the  problem.  Progress  toward  completion  can  then 
begin  again. 

If  communications  haw  been  effective,  the  Con¬ 
tractor  and  the  Government  are  now  a  team.  The 
final  milestone  has  resulted  in  victory  for  both, 
contract  completion.  The  Government  has  obtained 
a  good  product  for  a  fair  price  and  the  Con¬ 
tractor  has  realized  a  reasonable  profit.  Ef¬ 
fective  communications  have  made  this  possible, 
and  both  have  achieved  success. 

CONCLUSION 

As  the  Procurement  Communic ation  game  progressed, 
neither  the  Government  nor  Industry  could  advance 
through  the  milestones  without  communicati ng  with 
the  other.  Milestone  #1  required  that  communi¬ 
cation  be  established  between  the  opposing  teams. 
Success  in  continuing  the  game  depended  upon 
awareness  by  Industry  of  Government  requirements 
and  awareness  by  Government  of  Industry's  tech¬ 
no  logical  gains. 

Milestone  £2  built  on  that  initial  communication 
baseline.  Through  effective  conwunication,  the 
Government  team  achieved  success  in  the  inning 
by  publishing  a  clear  and  concise  RFP  while  pro¬ 
viding  all  offerors  the  same  information  and  the 
Industry  team  achieved  success  by  submitting  a 
responsive  and  real i st ical ly  priced  proposal. 

Milestone  H  3,  Contract  Award,  marked  the  mid¬ 
point  in  the  game.  Negotiations  required  that 
effective  verbal  communication  be  established  by 
each  team  so  that  the  Government  could  assume 
t ha t  its  needs  would  be  successfully  met  and  In¬ 
dustry,  now  the  Contractor,  could  assure  that  the 
award  was  deserved. 

Post  Award  required  effective  communication  on 
the  informal  as  well  as  the  formal  level  by  both 
sides.  Success  in  Milestone  #4  was  keyed  to 
communication  by  all  team  members  to  each  other 
and  to  their  counterparts. 

When  the'  two  sides  reached  the  final  milestone, 
each  had  achieved  its  goal.  The  Government  had 
its  good  product  at  a  fair  price  and  the  Con¬ 
tractor  had  its  reasonable  profit.  Playing  the 
game  and  winning  required  an  exchange  of  infor¬ 
mation  that  was  thoroughly  understood  by  both 
teams.  Effective  communi cat  ions  were  an  essen¬ 
tial  key  to  success  in  the  game. 
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ABSTRACT 

^Mn  order  to  create  a  realistic  learning  environment  for  future  acquisition  managers  at  the 
Defense  Systems  Management  College,  a  series  of  20  case  studies  and  3  decision  exercises  have 
been  developed  to  provide  a  scenario  as  close  as  possible  to  the  real  world  of  weapons  systems 
acquisition  management.  A^womb-to-tomb^-approach  is  taken:  i.e.,  a  mythical  weapon  system, 
System  X,  begins  its  life  in  mission-area  analysis;  has  a  gestation  period  culminating  in  a 
Justification  for  Major  System  New  Start;  then  a  development,  production  and  deployment  life;  and 
finallv  is  retired  and  disposed  of.  The  system  acquisition  life  cycle  is  the  background  structure 
on  which  the  storv-line  for  these  cases  is  maintained.  The  Decision  Exercises  utilize  the  same 
story-line  as  the  cases;  they  provide  the  student  an  opportunity  to  make  systems 


acquisition  decisions,  then  live  with  the  consequences  of  these  decisions. 


DEFENSE  SYSTEMS  MANAGEMENT  COLLEGE 


The  mission  of  the  Defense  Systems  Manage¬ 
ment  College  is  "...to  educate  acquisition  profes¬ 
sionals  and  conduct  research,  to  support  and  improve 
Defense  systems  acquisition  program  mangement."^) 

The  Defense  Systems  Management  College 
(DSMC)  is  located  in  five,  three-storv  buildings  and 
includes  a  400-seat  auditorium  on  Fort  Belvoir,  Va., 
20  miles  south  of  Washington,  D.C.  Sixty  civilian  and 
military  faculty  members  conduct  the  courses.  Their 
ettorts  are  complemented  by  guest  lecturers  from 
government,  industry,  and  the  academic  com¬ 
munities. 

The  College  program  includes  the  20-week 
Program  Management  Course,  numerous  short 
courses  in  specific  functional  areas,  and  several 
cxecutive-levcl  courses.  The  courses  are  intended  to 
introduce  the  student  to  the  world  of  systems  acqui¬ 
sition,  and  to  prepare  him  to  function  effectively 
within  it.  The  content  of  each  course  is  continuously 
monitored  and  altered,  as  necessary,  to  reflect 
changing  real  world  conditions.  New  short  courses 
are  developed,  usually  in  answer  to  the  needs  of  a 
specific  management  group  within  the  Department  of 
Defense.  The  curriculum  is  updated  to  keep  abreast 
of  current  management  practices  and  to  lead  the 
development  of  new  methods.  Each  faculty  member 
maintains  a  close  liaison  with  the  military  depart¬ 
ments,  other  educational  institutions,  industry  and 
business  organizations,  and  professional  societies. 

The  environment  of  defense  systems  acquisition 
is  an  ever-changing  mosaic  of  requirements, 
budgetary  constraints,  technological  capabilities,  and. 
political  and  strategic  considerations.  Preparing  the 
manager  to  deal  effectively  within  this  environment 
requires  a  dynamic  educational  program  that  blends 
abstract  concepts  with  real  world  experience.  The 
courses  offered  by  DSMC  are  designed  to  respond  to 
this  need.  (2) 

Program  Management  Course 

The  core  of  the  DSMC  program  is  the  Program 
Mangement  Course  (PMC).  It  is  attended  by  military 
and  civilian  students  from  all  services,  as  well  as 
representatives  from  defense  industry.  The  PMC  is 
offered  to  200  students  twice  a  year  and  includes 
courses  in  the  functional  area  of  program  manage- 


ment:  business,  technical,  organizational,  and  policy. 
To  achieve  integration  among  the  various  functional 
courses,  a  series  of  management  case  studies  and 
computer-based  decision  exercises  are  utilized. 

PMC  Students 

The  Program  Management  Course  is  generally 
restricted  to  military  officers  in  grades  0-3  to  0-5, 
DOD  civilians  in  grades  GS-11  through  CS-14,  and 
industry  personnel  identified  by  their  companies  as 
candidates  for  senior  management  positions.  These 
are  the  norms,  but  requests  for  exceptions  are 
reviewed  by  the  DSMC  admission  committee. 

In  addition  to  meeting  grade  requirements, 
PMC  students  must  be  DOD  personnel  who  occupy— 
or  who  have  been  selected  to  occupy— intermediate 
management  positions  in  program  offices,  functional 
offices  supporting  program  offices,  or  in  higher- 
echelon  offices  supervising  program  management. 
Persons  in  equivalent  program  management  positions 
in  other  federal  agencies  or  defense  industry  are  also 
selected  to  attend  the  Course. 

The  typical  Program  Mangernent  Course 
student  is  male,  married,  and  38  years  old.  He 
averages  14  years  of  military  or  government  civil 
service,  and  is  a  Major  or  a  C.S-12.  A  majority  of  the 
PMC  students  hold  graduate  degrees,  and  have  from 
3  to  7  years  experience  in  some  aspect  of  acquisition 
management. 

The  School  of  Systems  Acquisition  Education, 
through  four  departments— Policy  and  Organization, 
Technical,  Business,  and  a  multi-discipline, 
integrating  Management  Laboratory-conducts  the 
Program  Management  Course  (Figure  1),  as  well  as 
the  short  courses. 


POLICY  DEPARTMENT 

ENGINEERING  DEPARTMENT 

o  PROGRAM  MANAGEMENT 
o  HOD  POLICY 
o  BEHAVIORAL  SCIENCE 
o  EITECTIVE  COMMUNICATIONS 

o  SYSTEMS  ENGINEERING 

0  TESTING 
o  PROnUCTION 
o  LOGISTICS 

BUSINESS  PPARTMENT 

INTEGRATION  DEPARTMENT 

o  CONTRACTS 
o  GOVERNMENT  FUNDING 
o  INDUSTRY  FINANCE 
o  ESTIMATING/CSCSC 

o  SYSTEM  CASE  STUDIES 
o  DECISION  EXFR  1IISE  SIMULATIONS 
n  STUDENT  DECISION  BRIEFINGS 

FIGURE  1  PROGRAM  MANAGEMENT  COURSE 
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The  Aquisition  Management  Laboratory 

The  Acquisition  Management  Laboratory  pro¬ 
vides  experiential  learning  opportunities  that 
integrate  course  material  presented  in  the  three 
functional  departments. (3)  The  m^dia  used  include  a 
series  of  case  studies  covering  the  acquisition  life 
cycle  of  a  weapon  system,  computer-based  decision 
exercise  simulations,  and  individual  student  program 
management  decision  briefings. 

S vs tern  X 


Systems  X  consists  of  a  series  of  inter-related 
case  studies  involving  a  hypothetical  weapon  system. 
The  cases  simulate  the  life  cycle  of  a  weapon  system 
through  the  concept  exploration,  demonstration  and 
validation,  full-scale  development,  and  pro¬ 
duct  cm/ deployment  phases.  System  X  provides  a 
realistic  base  for  x he  -discussion  of  typical  problem 
areas  encountered  in  program  management.  Group 
( >-6  individuals)  analyses  of  case  material  are  made, 
alternatives  are  studied,  and  u  management  position 
derived.  The  analyses  are  followed  bv  section 
(U  groups)  discussions,  lead  by  a  faculty  case  leader, 
that  are  intended  to  focus  on  tlx?  relevant  issues  and 
provide  insight  to  the  best  possible  course(s)  of 
action. 


Students  work  in  heterogeneous  groups  designed 
with  student  interaction  in  mind.  They  are  not  afraid 
to  present  their  views  about  situations  created  in 
cases  or  simulations,  and  to  get  involved  solving 
acquisition  problems  with  classmates. 


DECISION  EXERCISES 

Students  participate  in  a  series  of  three 
Decision  Exercises  during  the  Program  Management 
Course.  The  exercises  attempt  to  simulate  the  real 
world  environment  of  acquisition  management. 

Concent 


The  Decision  Exercises  emphasize  issues  and 
dile. nmas  concerning  selected  tasks  possible  in  a 
program  office  during  a  phase  of  the  acquisition.  The 
student  work  group  is  asked  to  achieve  consensus 
regarding  decisions  about  these  issues. 

There  are  three  such  exercises,  each  intended 
to  reinforce  the  lessons  and  concepts  learned  during 
classroom  lectures  and  seminars,  as  well  as  System  X 
cases.  Decision  Exercise  I,  for  example,  is  a  com¬ 
puter  simulation  paralleling  the  first  six  System  X 
cases.  The  students  establish  a  schedule  of  activities 
for  the  Concept  Exploration  phase  of  a  weapon 
systems  acquisition;  then  enter  the  schedule  into  a 
computer,  and  manage  the  acquisition  of  the  weapon 
system;  then  enter  the  schedule  into  a  computer,  and 
manage  the  various  activities  as  the  system  pro¬ 
gresses  through  the  phase.  Decision  Exercise  II 
addresses  the  Demonstration  and  Validation  phase, 
with  Decision  Exercise  HI  addressing  Full-Scale 
Development  and  Production. 

The  desired  learning  objectives  of  the  Decision 
Exercise  are: 

o  To  enable  the  student  to  objectively 
analyze  and  quantify  the  complex  issues  involved  in 
weapons  system  acquisition. 


o  To  provide-  an  understanding  of  the 
inter-relationship  among  organizations  that  play  a 
role  in  the  process. 

o  To  exercise  the  student  to  make  timely 
decisions  that  consider  the  complexities  of  these 

inter- relationships.  (4) 

Purpose 

The  first  Decision  Exercise  simulates  the  Con¬ 
cept  Exploration  Phase  activities  from  the  submittal 
of  the  [Justification  for  Major  System  New  Starts 
(JMSNS)  with  the  Program  Objective  Memorandum 
(POM)  to  Milestone  1,  the  first  meeting  of  the 
Defense  System  Acquisition  Review  Council 
(DSARC  I). 

Decision  Exercise  I  Overview 


The  Decision  Exercise  is  designed  to  be  com¬ 
pleted  within  a  six-hour  period,  with  a  minimum  of 
instructor  supervision.  The  first  one  and  one-half 
hours. .is  devoted  to  an  introductory  lecture  and  group 
preparation  (Pre-Exercise  Preparation);  then  four 
hours  to  run  the  Decision  Exercise  program;  followed 
be  a  half-hour  for  debriefing  and  discussion.  The 
introductory  lecture  covers  the  desired  learning 
objectives,  group  organization,  allocation  of  time, 
and  homework  required. 

The  student's  goal  in  this  Decision  Exercise  is 
to  get  to  DSARC  I  on  time,  within  budget,  and  with 
the  "best"  concept(s)  for  continuation  into  the 
Demonstration  and  Validation  phase.  Throughout  the 
simulation,  the  student  team  is  faced  with  decisions 
that  impact  on  schedule,  budget,  and  personnel. 
Decisions  are  required  in  all  the  major  areas  of 
concern  to  a  program  manager. 

Individual  Preparation 

Prior  to  beginning  the  Decision  Exercise,  each 
student  completes  four  activities  outside  of  class. 
They  are  required  to: 

1.  Interpret  the  Justification  for  Major 
System  New  Starts  (JMSNS).  The  student  must  select 
from  a  list  of  those  factors  they  believe  best  describe 
the  deficiency  (Figure  2).  The  student  group  will 
weight  each  factor  selected  to  the  degree  of  relative 
importance. 
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FIGURE  2  POSSIBLE  EVALUATION  FACTORS 

2.  Prepare  a  Program  Objective 
Memorandum  (POM)  input.  Forthe  purpose  of  this 
requirement  the  student  must  submit  a  5- year  budget 
for  his  program.  Resources  made  available  to  hltn 
are  a  cost  estimating  relationship  (CER),  a 
comptroller  report,  OMB  inflatibn  Indexes,  and  an 

independent  cost  estimate  (ICE). 
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,  best  available  copy 


3.  Determine  project  office  staffing  require¬ 
ments,  i.e..  the  number  and  mix  of  military/civilian 
and  technical/management  personnel  for  the  phase 
(Figure  3). 


program  office 
(Figure  4). 


Preparation 
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FIGURE  3  PROGRAM  OFFICE  STAFFING 


4.  Schedule  the  major  tasks  and  activities 
from  the  beginning  of  the  Concept  Exploration  Phase 
until  Milestone  1.  The  activities  in  the  phase  have 
been  organized  into  17  different  tasks,  some  indepen¬ 
dent,  and  some  dependent  on  others.  Students  are 
askea  to  organize  and  schedule  these  tasks  and 
activities,  then  respond  to  prompting  oi  leading 
questions,  ranging  from  establishing  and  staffing  the 


In  preparation  for  the  simulation,  students 
participate  in  a  sample  computer  simulation  exercise 
‘•’signed  to  acquaint  them  with  the  use  of  the  com¬ 
puter  terminals,  available  commands,  ana  situations 
of  the  type  to  be  expected.  Immediately  following, 
students  are  squired  to  achieve  concensus  on  the 
four  area:.  they  prepared- -staffing,  tasks, 
interpretation  ol  the  JMSNS,  and  the  POM.  This 
forms  the  basis  for  the  group  input  at  the  beginning 
of  the  exercise. 

The  Exercise 

When  the  exercise  begins,  a  3MSNS  for  a  Long 
Range  Penetrator  has  been  approved  by  tne  Secretary 
o(  Defense  and  the  students  have  been  assigned  as 
"joint"  program  managers.  They  already  have  been 
given  office  space,  hired  secretarial  help,  and  have  an 
initial  staff  of  five  military  personnel  (2  managerial 
and  3  technical).  The  initial  guidance  is  to  determine 


FIGURE  4  DECISION  EXEVCISh  ACTIVITY  SCHEOl  LE 


the  best  solution  to  satisfy  the  documented 
deficiency.  The  DSARC  is  scheduled  !8  months  irom 
the  start  time. 

The  distinction  between  military  and  civilian 
personnel  staffing  is  made  to  simulate  the  time  and 
cost  associated  with  staffing  a  project  office.  Each 
task  to  be  accomplished  requires  a  preset  number  of 
one  of  those  specialties  (Figure  4).  Prior  lectures  and 
seminars  will  "'ave  given  the  students  an  idea  of  w-hat 
might  be  required,  in  terms  of  people,  for  each  task. 

For  the  purpose  of  the  exercise  each  team  is 
given  a  project  budget  and  an  initial  allocation  of 
funding  for  outside  support,  e.g.  contractors,  govern¬ 
ment  labs,  or  outside  agencies.  The  student  team  is 
first  asked  to  enter  their  schedule  of  tasks  and 
activities  into  the  computer.  Next,  a  particular  task 
is  selected  by  the  student  team;  the  simulation  will 
either  provide  information,  request  a  decision,  or 
pose  a  problem.  The  simulation  determines  the 
number  of  days  required  to  complete  the  team's 
selected  action.  The  clock  for  that  task  is  then  set 
ahead  by  a  specified  number  of  days,  dependent  on 
the  action  taken.  No  further  activity  can  take  place 
in  that  task  until  the  simulation  moves  forward  to 
that  time  period. 

For  example,  the  task  Establish  and  Staff  Pro¬ 
gram  Office  is  initiated  early  in  the  simulation. 
Students  are  informed  of  the  personnel  cost  and  time 
delays  involved  in  staffing  the  program  off?  e  with 
military  and  civilian  personnel.  The  simulation  asks 
for  the  number  of  military  and  civilian  personnel  the 
program  manager  wants.  The  team,  through  its 
response,  is  advised  of  the  number  of  days  that  will 
be  required  by  headquarters  for  processing  approval 
of  the  personnel  requests. 

Sample  Print-out 

Establish  and  Staff  Program  Office 

'tour  program  management  office 
(PV.O)  is  in  the  process  of  being 
organized.  Your  present  staft  consists  of 
five  military  personnel.  They  are  your 
technical  advisor,  two  technical  as¬ 
sistants,  and  two  management  assistants. 

At  this  time  you  should  determine  addi¬ 
tional  staffing  requirements  (exclusive  of 
secretarial  help)  up  to  Milestone  1.  Once 
the  staffing  requirements  are  determined 
vou  will  be  required  to  submit  a  personnel 
request.  Your  budget  expenditures  for 
civilian  personnel  will  be  $32K  per  person 
per  year.  There  are  no  budget  costs  for 
military  personnel.  Civilian  personnel  re¬ 
quested  may  begin  arriving  in  as  little  a* 
two  weeks.  Military  personnel  will  be 
assigned  within  six  to  ten  weeks  after 
they  aic  requested.  At  this  time  you 
should  have  your  staffing  plans  tor  this 
phase  firm. 

You  may  now  enter  your  personnel 
request. 

The  program  asks  the  stuJent  team  to  respond 
to  prompts  with  the  numbers  of  military  technical, 
military  management,  civilian  technical,  and  civilian 
management  personnel  thev  want  to  request.  The 
next  question  provides  them  with  an  opportunity  to 
correct  their  request. 


AM  C  vuu  aciuiivu  ...v..  -  , - - 

numbers  you  entered? 

Respond  yes  or  no. 

If  they  respond  "no,"  the  students  are  given  an 
opportunity  to  revise  the  numbers  of  personnel 
entered  earlier.  If  they  answer  "ves."  the  next 
question  is  displayed: 

Your  staff  has  prepared  a  personnel 
request  to  be  sent  through  appropriate 
channels.  A  response  to  this  request  is 
expected  within  a  month.  To  send  the 
request  press  "Return." 

The  student  is  then  advised  of  the  number  of 
working  days  until  Headquarter's  review  of  the  per¬ 
sonnel  request  wi  1 1  be  completed,  at  which  time  the 
following  message  is  displayed: 

Your  personnel  request  has  been 
processed  and  the  following  numbers  of 
civilian  and  military  personnel  have  been 
approved: 

The  numbers  of  pel  sonnel  approved  in  each 
category  are  displayed,  along  with  civilian  personnel 
costs  for  the  next  IS  months.  These  personnel  costs 
are  deducted  from  the  approved  budget. (5^ 

Equipment 

There  are  16  classrooms  equipped  for  tlie 
Decision  Exercise  simulation;  six  students  are 
assigned  per  room.  Each  simulation  room  contains 
.'vo  1200  BAUD  Hazeltine  Executive  SO  Model 
30  "smart"  CRT's  with  keyboard  terminals,  and  one 
Centronics  Model  704  impact  punter.  The  terminals 
are  configured  so  that  either  keyboard  may  be  used 
to  respond  to  the  prompts  from  the  program.  All 
three  devices  (2  CRTs  and  1  printer)  display/print 
information  simultaneously.  In  addition  to  the  equip¬ 
ment  in  each  simulation  room,  four  terminals  ann 
printers  are  available  in  the  faculty  observation 
center.  Each  terminal  can  be  used  to  remotelv 
monitor  student  operations  in  any  of  four  classrooms. 

The  Defense  Systems  Management  College  is 
currently  under  contract  to  the  McDonnell  Douglas 
Automation  Company  (MCAUTO)  time-sharing 
system  in  St.  Louis,  Mo.  A  CDC  Cyber  175  computer 
is  utilized.  Each  Decision  Exercise  occupies  40-5QK 
words  of  core  memory;  the  program  is  written  in 
FORTRAN  IV. 


Student  Team  Organization 


The  student  team  is  advised  to  regularly 
alternate  terminal  operators  and  not  to  center  group 
discussion  activities  around  tlx*  terminal.  It  is  also 
recommended  that  one  member  keep  track  of  the 
progress  of  the  Decision  Exercise  tasks  so  the  group 
can  anticipate  which  ones  to  proceed  to  next.  The 
faciliatator’s  role  during  the  exercise  is  essentially  a 
passive  one;  he  monitors  the  progress  of  each  group 
and  may  stimulate  discussion,  but  he  does  not  get 
involved  in  the  decisions. 


Debriefing 


The  student  team  log  file  contains  a  history  of 
the  interaction  between  the  student  and  the  system 
during  the  running  of  the  Decision  E.  *rcise.  It  is 
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designed  to  provide  the  instructor  with  a  summary  or 
the  completed  exercise;  it  may  be  accessed  whenever 
the  student  group  has  logged  ofi  the  terminal.  It 
records  che  actual  and  ideal  start  and  finish  dates,  as 
well  as  elapsed  time.  A  log  is  maintained  of  the 
budget  and  personnel  used  by  tiie  student*  team  com¬ 
pared  with  the  ideal.  Additionally,  student  decisions 
made  on  key  questions  or  problems  which  affect  cost, 
schedule,  or  technical  performance  are  recorded.  An 
opportunity  is  provided  for  students  to  list  comments 
directlv  from  their  terminal  as  the  exercise 
progresses.  The  informaMon  in  the  log  f;,e  is  used  by 
the  instructor  to  debrief  the  student  team.  In  the 
debriefing  the  student  team  is  given  feedback  on  how 
well  they  did  in  the  exercise. 

CONCLUSION 

Based  on  our  experience  in  conducting  case 
studies  and  decision  exercises  at  the  Defense  Systems 
Management  College,  we  have  found  that  students 
encounter  a  classical  dilemma.  They  usually  pass 
through  the  phase  of  (1)  being  overwhelmed  initially 
by  case  study  and  decision  exercise  material, 
(2)  realizing  that  one  must  pool  efforts  with  con¬ 
temporaries  in  order  to  succeed,  and  (3)  eventually 
coming  to  the  conclusion  that  the  student  has  as  good 
an  answer  as  the  instructor .(6)  Students  leave  the 
Defense  Systems  Management  College  having 
improved  their  skills  through  functional  instruction, 
as  well  as  by  experiencing  the  weapons  system 
acquisition  management  environment  through 
realistic  case  studies  and  decision  exercises. 
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ABSTRACT 


-  Flight  simulators  are  being  used  to  an  ever  greater  degree  to  train  combat  related 
skills.  The  Air  Force  Human  Resources  Laboratory  (AFHRL)  has  been  tasked  with  determining 
the  effecti veness  of  simulator  training  and  providing  guidance  as  to  how  to  train  for  combat 
in  a  simulator.  In  order  to  provide  these  answers,  high  fidelity,  realistic  combat  simulation 
mus+  be  accomplished.  Using  the  Advanced  Simulator  for  Pilot  Traininq  (ASPT),  techniques 
have  been  developed  for  the  generation  of  realistic  combat  environment  scenarios.  These 
techniques  were  used  to  develop  an  environment  that  closely  models  the  Tonopah  range  at 
Nellis  AFB,  Nevada,  a  range  that  is  often  used  for  REDFLAG  exercises.  Advanced  database 
model  nig  techniques  were  used  to  create  the  geographical  features,  cultural  features,  and 
provide  low-level  cues  utilizing  the  maximum  capability  of  the  ASPT  image  generating  system. 
The  environment  had  numerous  threats  including  surface-tn-ai r  missiles  and  anti -ai rcraft 
artillery.  The  pilot  could  interact  with  this  environment  in  the*same  manner  that  he  would 
interact  with  a  rea1  combat  environment.  Through  the  use  of  Radar  Homing  and  Warning  System 
(RHAWS)  and  the  visual  environment,  t+i^^ilot  could  determine  the  location  of  potential 
threats  nnd  targets.  The  p i  1 o f  could  attack  and  destroy  any  target  or  threat  within  the 
environment  and  he  could  be  "killed'1  by  jany  threat.  The  environment  simulation  techniques 
that  have  been  developed  are  very  flexible  and  therefore  the  REDFLAG  simulation  can  be 
quickly  adapted  to  provide  new  scenarios'* 


INTRODUCTION 

There  is  a  critical  need  in  the  Air  Forcfe 
for  realistic  combat  training.  Studies  have 
shown  that  if  an  aircrew  member  can  survive  his 
first  10  flights  in  the  combat  environment,  his 
chances  of  survival  are  dramatically  increased. 
These  first  10  sorties  represent  the  learning 
phase  for  the  combat  pilot.  The  Air  Force- 
currently  trains  pilots  in  combat  skills  through 
the  various  exercises  that  take  pl«ce  each  year. 

There  are  certain  inherent  disadvantages  to 
these  exercises.  First,  there  is  the  loss  of 
pilots  and  aircraft  due  to  accidents.  Second., 
there  is  a  tremendous  cost  associated  with 
conducting  the  exercises.  Third,  only  a  small 
percentage  of  the  operational  pilots  get  an 
opportunity  to  compete  in  an  exercise  at  any  one 
time.  And  last,  the  exercises  do  not  constitute 
a  continuous  training  but  rather  an  occasional 
test  of  previously  acquired  skills.  Flight 
simulators  are  not  faced  with  these  problems; 
however,  the  simulator  has  been  faced  with  the 
problem  of  developing  a  realistic  combat 
simulation. 

Engineers  and  Behavioral  Scientists  with 
the  Air  Force  Human  Resources  Laboratory  at 
Williams  AFB,  Arizona,  have  been  investigating 
the  simulated  combat  environment.  The  primary 
tool  used  in  this  work  has  been  the  Advanced 
Simulator  for  Pilot  Training  (ASPT)  located  at 
Williams  AFB.  The  ASPT  has  two  cockpits,  an 
A- 1 0  and  an  F - 1 6 .  The  A- 1 0  was  used  'or  this 
project.  For  details  on  the  ASPT  A- 1 0 
simulator,  see  Appendix  A.  As  the  focus  of 
research  being  performed  on  the  ASPT  has  shifted 
to  the  area  of  combat  skills,  techniques  have 
been  developed  that  allow  for  the  simulation  of 
realistic  combat  environments.  The  simulated 


REDFLAG  described  in  this  document  was  the  first 
application  of  these  techniques  in  direct 
support  of  simulator  combat  training  research. 
The  scenario  chosen  for  the  REDFLAG  research 
stuoy  was  compa^de  to  a  mission  the  s-’bject 
pilots  f lew  at  an  actual  REDFLAG  exercise.  The 
intention  of  the  study  was  to  collect  data  that 
would  provide  a  means  of  determining  if  there 
was  a  correspondence  between  the  subjects' 
performance  in  a  simulated  REDF1  *\G  mission 
profile  and  their  performance  in  the  actual 
REDFLAG  exercise  mission. 

VISUAL  ENVIRONMENT 

Simulated  environments  are  flat  earth 
models  cf  either  real  or  imaginary  places.  Real 
terrain  features  such  as  mountains,  dry  lake 
beds,  roads,  and  streams  are  depicted  in 
abstract  form  in  the  simulated  visual 
environment.  Mountains  are  depicted  by  prisms 
shaped  to  correspond  to  the  general  contours  of 
the  real  mountains  they  are  intended  to 
represent.  Dry  lake  beds  and  fields  are 
simulated  by  irregular  areas  of  contrast  on  the 
ground's  surface  and  are  not  associated  with  a 
change  in  elevation.  Currently,  roads  are 
always  composed  of  straight  segments;  curves  are 
very  expensive  in  terms  of  the  image  generation 
resources  required  to  depict  them. 

The  REDFLAG  visual  environment  is  a 
simulation  of  one  of  the  Nellis  Air  Force  Base 
ranges  located  southeast  of  Tonopah  and 
northwest  of  Las  Vegas  (see  Appendix  B).  Figure 
1  is  a  partial  Tactical  Pilotage  Chart  of  the 
area.  Figure  2  is  a  nap  of  the  simulated  visual 
environment  with  the  locations  of  the  targets 
indicated.  The  environment  modelled  induces 
command  and  control  posts  depicting  real  sized 
radar  units,  SAM  and  AAA  sites,  and  tank 
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AAA  =  Anti-Aircraft  Artillery 

CP  =  Command  Post 

FT  =  Friendly  Tanks 

SAA  =  SAM  Type  1 

SAB  =  SAM  Type  2 

SAC  =  SAM  Type  3 

TG  =  Tank  Group 

Map  of  the  Visual  Simulation 
Figure  2 

formations.  Figure  3  is  the  visual  simulation 
of  a  SAM.  Figure  4  is  the  visual  simulation  of 
a  Russian  tank. 

These  vehicles  are  surrounded  by  a  sea  of 
random  shaped  rock-like  objects,  each  SC  feet 
high.  The  REOFLAG  scenario  requires  the  pilot 


to  fly  a  port''”  of  his  mission  in  the  low 
alti  uie  regime  (see  Appendix  C).  The  vertical 
objects  p*ovide  adequate  visual  altitude  cues 
and  to  not  exceed  the  displayatle  eage 


SAM  Threat 
Figure  3 

limitation.  A  continuing  problem  is  getting  the 
maximum  number  of  cues  using  the  least  number  of 
computer  edges.  Efficient  3-D  cues,  each  having 


Russian  Tank 
Figure  4 

six  edges,  were  used.  The  REDFLAG  task  required 
the  pilot  to  maneuver  his  aircraft  over  a  large 
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area  of  ground.  The  vertical  cues  are  placed  in 
areas  that  would  be  used  for  ingress,  egress, 
and  threat  evasion,  and  are  placed  2000  feet 
apart,  give  or  take  25%.  Making  the  average 
spacing  2000  feet  enabled  the  objects  to  cover  a 
large  area.  The  2b%  variation  is  used  to  make 
the  objects  appear  randomly  positioned. 

Finding  a  target  using  the  6  arc-minute 
ASPT  visual  system  is  a  problem  when  the  target 
is  the  size  of  a  tank.  The  level  of  detail 
feature  is  used  in  an  effort  to  correct  this 
problem.  When  approximately  2-1/2  miles  from  a 
tank,  the  least  detailed  version  becomes 
active.  At  about  1-1/2  miles,  a  more  detailed 
version  replaces  the  least  detailed  version. 

The  most  detailed  tank  becomes  active  at  about 
one  mile  range.  The  least  detailed  version  is 
made  twice  the  size  of  a  regular  tank.  The 
intermediately  detailed  version  is  50%  larger 
than  an  actual  tank.  The  mosi  detailed  version 
is  realistically  sized.  The  other  vehicles  in 
the  environment  are  handled  the  same  way.  The 
less  detailed  versions  use  fewer  edges.  This 
procedure  made  the  targets  large  enough  to  be 
located  from  a  somewhat  realistic  range. 


Aerial  View  of  Simulated  Environment 
Figure  5 

The  basic  ground  is  a  medium  gray  while 
mountains  are  shaded  darker.  Targets  are 
generally  medium  dark.  The  vertical  altitude 
cues  and  the  roads  are  very  dark.  Dry  lake  beds 
are  depicted  using  a  gray  shade  that  is  lighter 
than  the  ground.  Figures  5  and  6  are  views  of 
the  simulated  target  area.  At  any  given  time, 
the  environment  will  use  approximately  150  edges 
for  mountains,  50  for  dry  lake  beds,  50  for 
roads,  500  for  vehicles  on  the  ground,  100  for  a 
SAM  inflight  (there  can  be  up  to  three  visible 
at  any  time),  and  as  many  of  the  remaining  edges 
as  possible  on  the  3-D  vertical  visual  cues. 

The  environment  is  modeled  to  depict 
accurately  the  actual  REDFLA6  area  as  nearly  as 
a  25GO  edge  capacity  permits.  Every  potential 


viewpoint  was  carefully  considered  to  see  that 
the  maximum  number  of  displayable  edges  were 
being  used.  The  visual  environment  simulation 
operates  synchronously  with  a  set  of  threat 
simulation  programs.  The  pilot  is  able  to 
locate  and  kill  a  target  while,  at  the  same 
t’me,  the  target  might  kill  him. 


Aerial  View  of  Simulated  Environment 
Figure  6 

THREAT  SIMULATION 

The  programs  which  drive  the  threat 
simulation  require  information  which  describes 
the  threats'  environment  in  a  fashion  consistent 
with  the  manner  in  which  they  operate.  Any 
specific  threat  site  has  its  own  perspective  and 
experiences  the  environment  only  in  terms  of 
what  it  can  "see"  from  its  point  of  view  and 
within  its  capabilities. 

In  the  "real"  world,  threats  "see"  their 
environment  through  the  information  brought  back 
via  returning  radar  signals,  infrared  signals, 
or  visual  line  of  sight  depending  upon  the 
system.  In  the  simulator,  this  information  must 
be  provided  in  an  entirely  different  way.  Since 
the  programmer  knows  where  the  threats  are  in 
the  environment,  the  areas  each  threat  will  be 
abie  to  "see"  can  be  predetermined,  given  the 
known  capabilities  of  the  threat.  In  combat 
environment  simulation,  the  limits  of  what  each 
threat  site  is  able  to  "see,"  i.e.  its  horizon, 
is  defined  in  ter«.,s  of  two  parameters  called  the 
maximum  effective  range  and  the  minimum  look 
angle.  The  meaning  of  maximum  effective  range 
is  readily  apramnt,  but  what  is  meant  by 
minimum  look  angle  requires  some  explanation. 

The  minimum  look  angle  is  found  by 
determining  the  angle  of  elevation  (measured 
from  the  ground  up)  below  which  a  threat  would 
not  receive  meaningful  information.  Two  types 
of  factors  influence  the  value  of  this  minimum 
look  angle:  system  induced  limitations  and 
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terrain  induced  limitations.  System  limitations 
include  the  mechanical  limits  of  the  arr.enna 
scan  and  its  susceptioi 1 ity  to  ground  return. 
Terrain  limitations  are  imposed  by  the  proximity 
of  mountains  to  the  threat  site.  The  presence 
of  a  mountain  within  the  line  of  sight  increases 
the  minimum  look  angle  for  all  ranges  beyond  tne 
mountain  in  that  sector. 

To  define  the  horizon  for  each  threat  site, 
a  polar  coordinate  "grid"  is  centered  at  each 
site.  Each  sector  of  the  grid  is  bounded  on  the 
left  and  right  by  azimuth  radials  and  the  near 
and  far  side  by  concentric  range  rings.  A 
single  value  for  the  site's  minimum  look  angle 
is  assigned  to  each  sector  of  the  grid.  This 
requires  that  when  a  mountain  contributes  to 
only  a  part  of  the  sector,  its  contribution  is 
averaged  on  a  proportional  basis  before 
determining  what  look  angle  value  to  assign  to 
the  entire  sector. 

The  accuracy  of  the  horizon  defined  in  such 
a  fashion  depends  on  the  interval  between  sample 
points  along  the  horizon.  Here  a  compromise 
must  be  reached  between  the  simulation's  realism 
and  the  available  simulator  resources.  If  a 
very  small  interval  is  chosen,  the  fidelity  of 
the  model  is  ve^y  good  but  the  computer  memory 
required  is  very  large.  Combat  simulation 
requires  that  the  correspondence  between  the 
visual  environment  and  the  threat's  horizon 
defining  data  be  close  enough  to  allow  the  pilot 
to  employ  terrain  maskiny  in  threat  avoidance. 
Since  the  aircraft  is  in  constant  motion,  it  is 
possible  for  an  approximation  of  the  threat's 
horizon  to  suffice  without  losing  realism. 


Threat  Simulation  Module  Structure 
Figure  7 

The  threat  simulation  software  is  organized 
into  a  series  of  modules  (see  Figure  7)  each  of 
which  performs  a  characteristic  function.  This 
structure  allows  easy  modification  of  the  method 
of  implementing  a  particular  function  without 
necessitating  that  large  portions  of  the 
supDorting  software  be  rewritten.  What  follows 


is  a  discussion  of  the  basic  functions  and 
methodology  of  the  major  modules. 

Since  the  threat  horizon  data  are  deter¬ 
mined  oy  location  in  a  specific  visual 
environment,  it  is  convenient  to  piace  these 
data  in  a  separate  module.  This  ’S  the  sole 
function  of  the  Threat  Environment  Data  Module, 
TEDM.  TEDM  is  an  interchangeable  module  which 
contains  all  data  that  could  be  interpreted  as 
scenario  dependent.  This  includes  the  location 
data  for  each  threat  in  the  visual  environment 
and  some  of  the  data  determining  the  operational 
characteristics  of  the  threat  as  well  as  the 
threat  horizon  defining  data  discussed  above. 
This  arrangement  allows  the  threat  simulation  to 
be  adapted  to  another  combat  scenario  with 
minimum  difficulty. 

The  logical  heart  of  the  threat  simulation 
is  the  Ground  Control  Intercept  (GCI)  module. 

It  uses  data  provided  by  TEDM  and  parameters 
defining  the  aircraft's  location  in  space  to 
detemine  which  threats  can  "see"  the  aircraft. 
This  is  accomplished  by  calculating  the  bearing, 
range,  and  look  angle  from  each  site  to  the 
aircraft's  current  position  and  comparing  them 
to  the  stored  data  that  define  the  radar  horizon 
for  that  site.  As  previously  mentioned,  the 
effect  of  local  terrain  is  incorporated  in  the 
radar  horizon  data  for  each  site.  If  the  pilot 
approaches  the  threat  using  terrain  masking, 
i.e.,  maintaining  low  altitude  and/or  keeping 
mountains  between  the  aircraft  and  the  threat, 
it  is  possible  to  get  quit^  close  to  the  threat 
without  its  becoming  active.  This  capability  is 
essential  to  realistic  air-to-ground  combat 
simulation. 

Orce  the  look  angle  test  is  satisfied,  the 
threat  site  is  activated.  Threat  activation 
consists  of  making  an  entry  defininq  the  nature 
of  the  site  and  its  status  in  an  array  located 
in  common  memory.  Data  entered  in  the  array 
tell  all  the  user  programs  the  status, 
location,  and  types  of  sites  that  are  active. 

The  nature  of  the  data  entered  in  the  active 
threat  array  depends  on  the  type  of  site. 
Basically  each  entry  contains  only  data  which 
are  needed  by  user  programs  to  drive  the  threat 
models.  S'te  type  and  location  are  data  that  do 
not  change  within  a  given  scenario;  however, 
site  status  is  highly  time  dependent. 

A  subsidiary  function  of  GCI  is  the 
invocation  of  user  programs  in  the  proper 
sequence.  Each  user  program  may  modify  the  data 
stored  in  the  active  threat  array  to  further 
prepare  it  for  use  by  other  programs 
downstream.  This  is  consistent  with  the  modular 
concept  by  keeping  all  functions  in  the  most 
appropriate  module.  For  example,  GCI  will  set 
the  status  of  the  Surface-to-Air-Missi le  site  in 
either  acquisition  or  track  and  will  determine 
which  of  two  otherwise  equally  active  sites 
should  be  given  priority  for  launch.  It  then 
lets  the  appropriate  subroutine  in  the 
Surf ace-to-Air-Missi le  (SAM)  module  determine 
when  to  actually  launch  the  missile  from  the 
site  GCI  has  selected. 

Once  all  sites  have  been  checked,  control 
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passes  to  the  countermeasures  (CM)  module.  It 
checks  for  release  of  chaff  and/or  flares,  then 
determines  the  effectiveness  of  the  counter- 
r.,earure  selected.  Effective  countermeasures 
influence  the  operation  of  the  S/M  or  Anti- 
Aircraft  A-'tillery  (AAA)  mod  les  by  denying 
updated  aircraft  position  and  velocity  vector 
data.  Countermeasure  effectiveness  can  be 
varied  in  many  ways  depending  on  the 
requirements  of  the  research  or  training.  In 
the  REDFLAG  simulation  only  chaff  was  available 
and  each  chaff  release  provided  3  seconds  of 
effective  chaff. 

Upon  completion  of  execution  of  the 
countermeasures  (CM)  module  GCI  calls  the  SAM 
module.  Actually  the  SAM  module  consists  of  a 
series  of  subroutines  each  of  which  is 
responsible  for  driving  a  model  of  a  particular 
type  of  surface-to-air  missile.  All  surface-to- 
air  missile  systems  are  currently  modeled  in  the 
same  manner,  i.e.,  the  basic  structure  of  the 
subroutine  is  the  same  for  all  models.  However, 
each  model  uses  different  values  for  critical 
parameters  such  as  maximum  speed  or  turn  rate  so 
that  the  performance  of  the  model  reflects  the 
capabilities  of  the  system  it  represents. 

Each  SAM  subroutine  performs  the  same  basic 
functions.  First  it  searches  the  active  threat 
array  for  threats  of  its  type.  Then  it  checks 
the  status  of  each  threat  found.  For  threat;  in 
the  aquisition  mode  no  further  action  is 
necessary.  For  threats  in  the  track  mode  the 
subroutine  attempts  to  align  the  missile  with 
the  line  of  sight  to  the  target  aircraft.  Once 
the  missile  is  aligned  it  is  launched  and  the 
subroutine  guides  it  on  a  course  intended  to 
intercept  the  aircraft's  flight  pa Lh .  While  the 
missile  is  in  flight,  the  subroutine  continually 
checks  the  missile’s  proximity  to  the  target  to 
detect  the  moment  of  closest  approach.  It  is 
assumed  the  warhead  will  detonate  at  closest 
approach.  The  distance  separating  the  mssile 
and  the  aircraft  at  closest  approach,  i.e.,  the 
miss  distance,  is  compared  to  the  kill  radius  of 
the  missile  warhead  to  determine  if  the  aircraft 
has  been  destro3'ed. 

Each  subroutine  in  the  SAM  module  is 
capable  of  handling  several  different  threats  of 
its  type  simultaneously.  An  exception  is  that 
only  one  missile  of  each  type  is  allowed  to  be 
in  flight  at  a  time.  This  restriction  is  a 
result  of  the  manner  in  which  the  REUFIAC 
threats  were  simulated.  In  the  simulated 
REDFLAG  three  moving  models  were  used  to 
represent  inflight  SAM  missiles;  one  dedicated 
to  each  type  of  missile  modeled. 

After  tne  SAM  module  has  executed  GCI  calls 
the  AAA  module.  In  the  simulated  REDFLAG,  time 
elapsed  since  the  aircraft  came  over  the 
threat's  horizon  is  the  primary  criterion 
governing  operation  of  the  AAA.  A  couple  of 
seconds  must  pass  before  a  site  will  recognize  a 
target.  Then  a  few  more  seconds  will  pass 
before  the  site  will  begin  firing.  A  kill  is 
determined  by  the  amount  of  time  the  aircraft 
remains  in  the  AAA  firing  envelope. 

Both  the  AAA  and  SAM  threat  modules  respond 
to  evasive  manuevers  performed  by  the  pilot. 


The  SAM  module  subroutines  react  to  aircraft 
manuevering  when  computing  target  intercept 
guidance  and  can  be  uut-manuevered.  The  time 
required  by  AAA  to  kill  the  aircraft  can  be 
increased  if  the  pilot  executes  a  high-G  turn 
while  receiving  fire. 

The  last  module  called  by  GCI  controls  the 
in-cockpit  Radar  Homing  and  Warning  (RHAW) 
display.  The  RHAW  informs  the  pilot  of  his 
exposure  to  threats  using  data  supplied  by  the 
GCI,  SAM  and  AAA  modules  via  the  active  threat 
array.  For  the  REDFLAG  simulation  a  generic 
RHAW  display  was  provided.  Each  active  threat 
was  shown  on  the  face  of  the  instrument  at  a 
clock  position  consistent  with  its  bearing 
relative  to  the  aircraft.  The  status 
(aquisition,  track,  etc.)  and  type  (SAM  or  AAA) 
of  each  threat  wa.  indicated.  However,  no 
information  indicating  the  highest  priority 
threat  or  the  specific  kind  of  SAM  was 
available  to  the  pilot. 

The  results  of  the  pilot's  weapon  attempts 
against  targets  are  determined  by  the  SCORING 
module.  The  SCORING  module  runs  independently 
of  CCI,  but  information  id  ntifying  killed 
threats  is  communicated  to  GCI  by  scoring  so 
that  killed  threats  are  deleted  from  the 
scenario.  The  REDFLAG  visual  environment 
contains  numerous  scorable  targets.  The  scoring 
module  must  determine  which  of  the  possible 
targets  the  pilot  is  attempting  to  hit  and 
report  the  results.  To  reduce  the  amount  of 
processing  required,  a  list  of  potential  targets 
is  *ormed  from  all  those  possible  during  the 
time  between  weapon  releases.  Dotential  targets 
satisfy  three  criteria: 

1.  They  have  not  been  killed  previously. 

2.  They  are  in  front  of  the  aircraft. 

3.  They  are  relatively  close  to  the 
ai rcraft. 

When  a  weapon  attempt  occurs,  SCORING 
selects  the  target  closest  to  the  weapon  impact 
point  from  the  list  of  potential  targets  as  the 
target  the  pilot  intended  to  hit.  If  the  weapon 
attempt  was  a  strafe  pass,  the  miss  distance 
relative  to  the  target  center  and  the  clock 
position  of  each  round  is  computed.  The  miss 
distance  is  used  to  determine  if  the  round  was  a 
hit.  At  the  completion  of  each  str0fe  pass,  the 
number  of  hits,  total  rounds  fired  and  thp 
distribution  of  rounds  fired  is  provided  on  a 
CRT  display  legated  ?t  the  simulator  operator's 
control  console.  For  bomb  releases  miss 
distance  and  c’ock  position  are  calculated  anc 
displayed. 

The  kill  criterion  in  effect  fo,r  the 
simulated  REDFLAG  was  one  strafe  hit  killed  any 
target.  A  hit  was  defined  as  a  round  passing 
within  5  feet  of  target  center.  A  bomb 
impacting  within  150  feet  of  any  target  would 
also  qual ify  as  a  kill. 

CONCLUSION 

Effective  combat  simulation  could  provide 
the  pilot  with  the  opportunity  to  learn, 
practice,  and  improve  his  combat  skills  in  a 
safe  and  cost-effective  manner.  The  methodology 
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outlined  in  this  paper  significantly  contributes 
to  the  capability  to  conduct  rese'rch  to 
determine  what  constitutes  effective  combat 
simulation  and  could  provide  the  framework  for 
operational  simulator  combat  training. 

APPENDIX  A 

ASPT/A-10  Description 

The  Advanced  Simulator  for  Pilot  Training 
(ASPT)  is  a  research  simulator  originally 
designed  with  a  full  mission  T- 37  capability.  A 
detailed  description  of  the  original  uevice  may 
be  found  in  Gum,  Albery,  and  Basinger,  1975 
(AFHRL-TR-75-59) .  One  cockpit  has  been  modified 
to  represent  an  A- 1 0  configuration  while  the 
other  cockpit  has  been  configured  as  an  F-16 
aircraft.  Neither  of  the  modified 
configurations  has  full  mission  capabilities. 
Both  systems  were  designed  to  have  necessary 
cockpit  and  aerodynamic  capabilities  to  support 
transition  flight  tasks  such  as  takeoffs, 
approaches  and  landings,  basic  navigation  tasks, 
and  conventional  ir-to-ground  weapons  delivery 
tasks.  Both  cockpits  are  being  continually 
modified  to  expand  the  capability  for  combat 
simulation  and  more  closely  simulate  actual 
aircraft  capabilities  end  characteristics. 

The  following  is  a  description  of  the 
ASPT/A-10.  The  visual  display  is  a 
monochromatic  computer  gene,  afed  image  displayed 
through  se  ‘en  CRTs  with  a  300°  horizontal  by 
+110°  vertical  field  of  view.  The  ASPT/A-10 
also  has  a  field  of  view  of  -20°  over  the 
nose,  -400  over  the  left  side,  and  -150  over 
the  right  side.  There  is  a  "G"  seat/suit 
capability.  The  cockpit  layout  was  designed  to 
duplicate  the  aircraft  in  most  major  respects. 
Aircraft  aerodynamics  provide  normal  ‘.light 
characteristics  throughout  the  aircraft  envelope 
. nd  Con  provide  characteristics  for  the  Manual 
Reversion  Flight  Control  System.  The 
aerodynamic  model  does  not  account  for  weapon 
weight  or  station  number  but  does  account  for 
weapon  drag.  All  flignt  and  engine  instruments 
are  operable,  including  the  HSI.  Ccmmunicatior s 
panels  are  static  mock-ups.  The  HUD  provides 
display*  for  manual  strafing  and  bombing.  The 
weapons  modeled  on  *he  A- 10  are  the  BDU  33  and 
the  30mm  cannon  at  the  high  rate  of  fire. 

APPENDIX  8 

ASPT  Visua.  System  Description 

The  Advanced  Simulator  for  Pilot  Training 
(ASPT)  Computer  Image  Generation  (CIG)  system 
stores  the  visual  environment,  defined  in  a 
three-dimensional  reference  system,  in  computer 
memory.  The  data  are  then  retrieved  and 
projected  as  a  perspective  image  on  a  series  of 
seven,  36- inch  1024  by  1024  CRTs. 

Visual  database  modeling  is  the  art  of 
defining  and  storing  tfrn  visual  environment  as 
numerical  data  in  computer  memory.  Maps, 
photographs,  and  scale  drawings  art  used  as 
source  data. 

The  CIG  is  composed  o.  vertices  defined 
mathematically  in  3-D  space.  These  vertices  ar-' 
connected  bv  edges.  The  edges  are  used  to  make 
faces.  The  faces  are  assigned  a  gray  shade  and 


used  to  make  2-D  or  3-D  objects.  The  objects 
are  used  to  construct  models.  The  models  are 
then  put  together  to  make  the  eny irnnmpnt. 

There  are  many  ’imitations  on  the  modeling 
of  the  visual  environment.  Faces  must  be  convex 
with  their  vertices  in  the  same  plane.  A  face 
may  have  up  to  16  vertices.  An  object  may  have 
at  most  32  vertices  and  16  faces.  Models  can 
hLve  no  more  than  15  objects.  In  a  visual 
environment  there  may  be  at  most  300, 0G0  edges, 
40,000  objects,  and  2000  models.  The  real-time 
visual  system  can  display  at  most  2560  edges, 

512  objects  and  200  models  in  the  frame  time  of 
1/30  second. 

The  visible  edges  per  system  limitation  are 
usually  encountered  first.  Because  of  extra 
edges  generated  when  edges  cross  window 
boundaries,  the  effective  maximum  edges  used  to 
make  a  visual  scene  should  not  exceed  2000.  If 
the  edge  limit  is  exceeded,  undesirable  visual 
?ffects,  such  as  priority  problems  and  portions 
of  the  scene  flashing  in  and  out,  occur. 

The  ASPT  visual  system  allows  three  levels 
of  detail  per  model.  The  switching  distance 
from  one  level  to  the  next  is  a  function  of  the 
aircraft  altitude  above  ground,  the  distance  to 
the  center  of  the  model,  and  the  size  of  the 
model . 

ASPT  uses  shades  of  gray  for  painting  the 
displayed  faces.  The  scale  goes  from  0  (very 
black)  to  63  (very  white). 

APPENDIX  C 

Low  Level  Modeling  Considerations 

Lack  of  adequate  visual  scene  detail  limits 
the  usefulness  of  the  Advanced  Simulator  for 
Pilot  Training  (ASPT)  computer  image  generation 
(CIG)  system  for  simulating  low  level  flight. 

The  visual  environment  modeler  should  optimise 
existing  capabilities  to  compensate  as  much  as 
possible  for  the  lack  of  scene  fidelity  to 
better  provide  for  the  low  level  task. 

Both  ground  texture  patterns  and  vertical 
objects  are  used  as  primary  visual  cues  by 
pilots  in  judging  aircraft  altitude  above  the 
ground.  Ground  texture  patterns  must  be  quite 
intricate  in  order  to  be  effective.  They 
therefore  use  a  large  number  of  computer  graphic 
"edges."  Ground  texture  can  be  used  effectively 
without  overloading  the  computers  if  a  preset 
flight  path  is  adhered  to  by  tiie  pilot. 

Vertical  cues  are  mere  effective  in  a  situation 
where  the  pilot  is  not  required  to  fly  a 
pre-defined  flight  path.  These  vertical  objects 
provide  a  better  altitude  cue  than  a  ground 
texture  pattern  composed  of  the  same  number  of 
edges.  A  vertical  cue  can  use  as  few  as  six 
edges.  The  cues  are  usually  made-  much  taller 
than  they  are  wide  to  enhance  their  vertical 
appearance.  These  cues  can  be  put  throughout  an 
environment  up  to  the  ooint  of  edge  overload. 

Environments  can  oe  set  to  work  quite 
efficiently  for  a  low  level  flight  with  models 
becoming  active  and  inactive  in  i  way  that  the 
system's  edge  limit  is  always  being  approached. 
The  distance  at  which  a  mouel  becomes  active  and 
then  switches  levels  of  detail  is  a  function  of 
the  aircraft  altitude  and  distance  from  the 
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center  of  a  model  and  the  size  of  tne  model. 
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If  the  pilot  must  raise  his  altitude  to 
turn  or  make  a  run  on  a  target,  the  increased 
altitude  can  cause  too  many  models  to  become 
active  and  thus  overload  the  system.  Under 
these  circumstances,  a  trade-off  must  be  found, 
usually  by  trial  and  erro^. 
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ABSTRACT 

Embedded  simu1atioo  embraces  the  concept  of  using  the  real  vehicle,  art ificallv  stimulated  in 
a  controlled  manner,  for  the  purpose  of  training  operators  in  the  use  of  the  vehicle  equip¬ 
ment.  All  the  advantages  of  a  controlled  training  environment,  safety,  malfunction  training, 
etc.  associated  with  dedicated  simulation  apply,  together  with  greatly  reduced  capital  costs 
and  added  operaticial  flexibility.  he  paper  describes  TRIAD,  a  prototype  simulation  complex 
developed  for  helicopter  pilot  training  that  incorporates  embedded  simulation  principles. 
TRIAD  romprises  three  major  components;  a  compute*-/ 1  ink  age/per  iphera  1  complex,  an  out-the- 
windnw  CIG  visual  display  system  and  a  Bell  206  Helicopter,  each  integrated  together  to 
demonstrate  the  feasability  of  embedded  simulation  for  pilot  training  while  maintaining 
certification  on  the  aircraft  configuration.  The  total  system  is  described  and  includes  the 
technical  approach,  an  assessment  of  achieved  performance,  cost  summary  and  suitability  for 
training  applications.  A  description  of  the  aircraft  modifications  is  presented,  detai ling 
how  they  are  achieved  without  compromising  aircraft  performance,  reliability,  or  availabil¬ 
ity.  '-Jhe  problems  solved  and  lessons  learned  during  this  program  are  discussed  together  with 
an  outline  of  future  goa’s  aimed  at  complete  system  simulation  with  minimum  aircraft  hardware 
adaptation .. 


INTRODUCTION 

The  purpose  of  this  study  was  to  investigate  a 
means  of  satisfying  the  training  needs  of  Off¬ 
shore  Logistics,  a  multi-type  helicopter  user  and 
operator.  In  an  environment  whe*-e  costs  and  the 
risks  to  train  in  the  actual  aircraft  have 
increased  significantly,  operators  have  utilized 
a  variety  of  devices  to  enhance  tneir  training 
and  safety  programs  in  a  cost  effective  way.  High 
technology  six  -axis  simulators  have  provided 
realistic  flight  training  but  are  very  expensive 
and  are  not  conveniently  accessible  to  some  seg¬ 
ments  of  the  aviation  industry.  This  is  espec¬ 
ially  true  where  several  aircraft  t,pes  are 
operated  by  ore  organization  with  the  cost  of 
purchasing  several  simulators  being  prohibitive. 

Embedded  simulation  is  the  concept  of  using  the 
real  vehicle  that,  when  stimulated  in  an  appro¬ 
priate  manner,  provides  the  illusion  of  being 
operated  in  the  real  world  environment  providing 
the  operator  or  pilot  corresponding  perceptive 
stimulus,  and  hence,  training  at  reduced  capital 
outlay. 


TRIAD  is  a  prototype  simulation  complex  that  is 
the  result  of  the  combined  efforts  of  Rediffusion 
Simulation  and  Offshore  Logistics  to  produce  a 
low  cost  training  system  which  incorporates  the 
sophistication  of  large  simulators  with  realistic 
flight  and  visual  cues.  The  helicopter  is  an 
ideal  candidate  to  investigate  embedded  simula¬ 
tion  concepts  in  that  it  is  of  relatively  small 
size,  contains  simple  aircraft  systems  and  is  a 
device  that  current  flight  simulators  do  not 
address.  Helicopters  are  more  difficult  to  learn 
to  fly  than  fixed  wing  aircraft  and  it  is 
believed  that  embedded  simulation  offers  a  safe 


and  inexpensive 
abinitio  training. 

approach  to 

recurrent 

and 

TRIAD  comprises 

three  major 

components; 

a 

computer/linkage/peripheral  complex,  an  out-the- 
window  CIG  Visual  Display  System  and  a  bell  206B 
Helicopter.  Figure  1  shows  the  interior  of  the 
Bell  206  cockpit  when  configured  for  simulation. 
The  portable  instructor  control  unit,  shown  on 
theleft,  employs  a  touch  sensitive  plasma  displav 
providing  both  input  and  output  control,  and  the 
out-the-w'ndow  scene  is  generated  by  the  NOVOVItW 
SP3T  CIG  Visual  system  employing  back  screen  pro¬ 
jection  display  techniques. 
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Offshore  Logistics,  as  well  as  being  a  major 
helicopter  operator,  also  supports,  maintains  and 
modifies  neli copters  of  all  types  and  has  the 
experience  and  expertise  to  design  and  implement 
interface  adaptations  that  are  necessary  to  arti¬ 
ficially  stimulate  instruments  and  controls  in  a 
real  aircraft  cockpit.  Red  if fusion  Simulation, 
for  more  than  thirty  years,  has  been  manufactur¬ 
ing  advanced  flight  simulators,  utilizing  real 
aircraft  hardviare  that  has  been  specifically 
adapted  for  use  in  a  ground  borne  simulator. 


The  use  of  real  aircraft  systems  in  a  non-real 
environment  demands  special  consideration.  In 
particular,  the  effect  of  induced  vibration  on 
the  engine  and  transmission  when  in  a  static  con¬ 
dition  and  its  influence  on  operating  life  is  of 
major  concern.  Because  of  this  it  was  decided 
no!  to  place  TRIAD  on  a  vibrating  platform,  but 
rather  investigate  other  ways  of  providing  the 
illusion  of  vibration,  a  necessary  part  of  any 
helicopter  simulation. 


OBJECTIVES 

The  primary  objective  is  to  be  able  to  convert  a 
cor.i.ied  operational  helicopter  into  a  full  hiqh 
rnleiuy  flight  simulator  while  maintaining  the 
current  aircraft  airworthiness  certification. 


■) rove  "Embedded  S  imu  1  a t  i on11  Co ncept 

Prove  that  embedded  simulation  has  a  practical 
application  in  providing  training  capability  in  a 
cost  effective  manner  by  interfacing  real  air¬ 
craft  hardware  to  a  simulation  computing  complex. 

liiatLU  Eroploy  a  high  fidelity  generic  hel¬ 
icopter  flight  simulation  model  with  some  adjust¬ 
ments  to  approximate  the  Bell  206B  performance 
characteristics .  Verification  data  will  be 
obtained  from  limited  flight  test  activity  on  a 
Bel  1  206 R  a i rcra f  t . 


Flight  Controls.  Install  positional  poten¬ 
tiometers  on-  the  throttle,  collective,  cyclic  and 
anti-torque  pedals  to  obtain  instantaneous  pilot 
control  movements. 


Engines  and  Rotor  Employ  a  representative 
model  of  the  Allison  250-C20  engine  and  rotor  to 
provide  approximate  Bell  206B  installed  perform¬ 
ance. 


n_ectr1cal  System.  Provide  olectrkai 
power,  from  an  external  source,  to  the  aircraft 
bu.-,os  simulating  the  aircraft  generator,  batter y 
and  400  Hz  converter  characterist ics. 

Hydrau 1 i c  System .  Provide  hydraulic  power 
t°  the  aTrcraf t  f rom  an  external  source  while 
interacting  appropri ately  with  engine  hydraulic 
pump  drive  character istics .  With  "boost"  on, 
provide  the  pilot  with  the  same  control  "feel"  as 
that  experienced  durinq  flight  in  the  real  air¬ 
craft. 

Navigation.  Interact  appropr i atel y  with  NAV 
selections  in  the  cockpit  (VOR,  ILS,'  ADF )  for 
selected  real  world  OCA  facilities. 

Simulation  Equipment.  Employ  proven  soft¬ 
ware  and  hardware  simulation  technology.  Use 
modular  hardware  interface  systems  to  permit 
future  expansion  with  minimal  redundancy.  Use  an 
available,  current  state  of  the  art  CIG  visual 
system,  interfaced  to  the  aircraft  fuselage  for 
maximum  f ield-of-view  and  have  the  flexibility  to 
adapt  to  different  aircraft  types. 

Total  Simulation  System  Validation. 


Determine  system  viability  by  monitoring  pilot 
performance,  reaction,  comment  and  acceptability 
of  the  embedded  simulation  configured  aircraft  as 
a  training  device. 


FIGURE  1  TRIAD  FROM  A  PILOTS  POINT  OF  VIEW 
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Additional  Objectives 


CIG  Visual  Display  Techniques.  Investigate 
optimum  f ield-of -view/winduw  placement,  real 
image  vs  virtual  collimated  image  presentation  in 
relation  to  pilot  eye  height  judgement  (recogniz¬ 
ing  that  the  display  system  must  be  readily 
adaptable  to  any  helicopter  window/fuselage  con¬ 
figuration)  . 


Sound/Vi  brat i on .  Investigate  the  effective¬ 
ness  "of  powerful  sound  stimulus  as  a  means  of 
introducing  realism  and  low  amplitude  vibration 
corresponding  to  that  induced  by  the  transmission 
and  rotor  dynamics. 


Simulation /Aircraft 
interface 


Interface. 


Develop  an 
for  rapid 


strategy  that  provides 
coupling  while  retaining  complete  aircraft  inte¬ 
grity  and  safety.  Adapting  aircraft  for  full 
simulation  use  and  return  to  operation  should  not 
exceed  two  hours. 


EQUIPMENT 


Summary 


Figure  2  shows  the  layout  of  the  helicopter,  the 
visual  display,  and  the  sound  system  in  a  room 
measuring  60  feet,  long  by  41  feet  wide.  The  sim¬ 


ulation  computing  complex  and  CIG  image  generator 
are  located  in  an  adjacent  room  measuring  20  feet 
by  20  feet.  The  display  screens  were  oriented  to 
the  pilot  side  and  provide  a  48  degree  vertical 
by  108  degree  horizontal  fiela  of  view.  Figure  3 
shows  the  display  screens  in  relation  to  the  hel¬ 
icopter  fuselage. 


Figure  5  shows  the  total  TRIAD  simulation  system 
and  the  major  lines  of  communication  between  sys¬ 
tem  units. 


Simulator  Computing  Complex.  The  TRIAD  sim¬ 


ulation  computing  complex  comprises  current  state 
of  the  art  equipment  and  software  designed  and 
developed  for  modern  day  simulators  operated  by 
the  major  airlines  throughout  the  world. 


The  major  simulator  computing  complex  units  are: 


General  Purpose  Computer.  SEL  32/275G  by 
Gould  Inc.,  S.c.L.  Computer  System  Division. 


Linkage  Interface  Type  1406  by  Rediffusion 
Simulation  Limited,  England 


Sound/Vibration  System  By  Labute  Professional 
Sound. 
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CIG  Visual  System.  The  NOVOVIEW  SP3T  Day/  indicators  ar'i  is  VFR  equipped  with  single  VOR 

Dusk7Nigiit  Visual  System,  by  Rediffnsion  Simula-  and  COMM  capability, 

tion  Inc.  was  chosen  for  the  simulator,  not  only 

because  u  is  a  day  light  sytem,  but  also  because  The  heliropter  configuration  used  tor  TRIAD 

it  provides  textured  surface  effects.  Helicopter  however,  is  IFR  equipped  with  Dual  Nav  and  Single 

pilot  coni rolab i 1 i cy  depends  to  a  large  extent  ADF  by  King  Avionics,  an  Air  Data  Computer  System 

upon  Lhe  pilots  ability  io  observe  small  aircraft  by  Intercontinental  Dynamics  Corporation,  a 

movements  relative  to  the  ground.  Textured  sur-  Remote  Attitude  Indicator  by  Jet  Electronics  and 

fa*.ef  will  greatly  enhance  the  perceptabi  1  i ty  of  Technology,  Inc.,  arid  a  full  complement  of  elec- 

these  small  movements.  tronic  driven  engine  instruments  by  Instrument 

Special  ities  Co.,  Inc. 

Bell  2063 .  The  Bell  2%B  (Jet.  Ranger)  is  a 

single  engine,  land  based,  utility-type  he  1 i cop-  Figure  4  shows  the  TRIAD  instrument  panel  lay- 

ter  with  a  standard  cor.  iquration  that  provides  out.  A  complete  list  of  instruments  and  indica¬ 
tor  one  pilot  and  four  passengers.  The  main  rotor  tors  are  shown  in  the  Tabic,  Figure  6. 

is  a  two-bladed,  semi-rigid,  see-saw  type  employ¬ 
ing  preconing  and  undersl ingir-g  to  ensure  smooth  All  the  instruments  are  electronically  driven 

operation.  The  helicopter  is  powered  by  a  model  with  the  exception  of  the  Turn  and  Slip  Indka- 

250 -C2C  shaft  turbine  engine,  manufactured  by  tor.  This  aircraft  instrument  is  replaced  by  an 

Allison  Division  of  General  Motors  Corporation.  A  electronically  driven  simulated  instrument  when 

typical  instrument  panel  has  fewer  tnan  twenty  the  helicopter  is  configured  for  training. 
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INSTRUMENT 


MANUFACTURER 


SIMULATION 

FUNCTION 


REQUIRED  INTERFACE 


COMMENT 


Remote  Attitude 

JET 

Pitch  and  Roll 

2  Synchro  Drive  Channels 

Indicator  RAI  303A 

Attitude 

Vertical  Gyro 

JET 

Drives  Marker 

Aircraft  Wiring  Only 

VG  208 

Flaq 

Airspeed  indicator 

I  DC 

Airspeed 

1  Analogue  Output 

38610-004 

-10  Volt  dc 

Alt imi ter 

I  DC 

Baro  Altitude 

Dual  Synchrodrives 

15790-202 

Vertical  Speed 

I  DC 

Rate  of 

None 

Part  of  ADC 

180138-950 

Cl imb/Descent 

System 

A  •  r  Data  Computer 

I  DC 

None 

None 

38620-001 

Standby  Altimeter 

IDC 

None 

None 

Not 

31400 

Simulated 

Standby  Airspeed 

I  DC 

None 

None 

Not 

36950 

Simulated 

Turn  and  Sl’p 

— 

Turn  Rate  and 

2  Analogue  Outputs 

Simulated 

Indicator 

Sideforce 

*10  Volt  dc 

I nstrument 

Radar  A 1 1 imi ter 

KING 

Height  Above 

1  Analogue  Output 

KNI  416 

Ground 

0  to  30  Volt  Amp  Module 

Horizontal  Situation 

KING 

Heading,  CDI , 

Compass  Drive  Card 

Indicator  (HSI) 

Glideslope 

0-200  m  volt. 

Localizer 

5-100  m  volt 

Current 

KNI  525A 

Warring  Flag 

Analogue  outputs 

Source  Drive 

Discrete  Output 

Radio  Magnetic  Indicator 

KING 

Heading 

Slave  From  HSI 

KNI  582 

VOR  Bearing 

16  bit  Serial  Data 

ADF  Bearinq 

Sin/Cos  Analoque  Output 

R.'IAV  Receiver 

KING 

DME  Se’ect 

NAV  Selected  Course/ 

KNS  81 

NAV  Selected 

Frequency  Board 

Course 

NAV/Comm  Module 

KING 

None 

None 

Not 

KX_165 

Simulated 

Comm  Module 

kTng 

None 

None  ~ 

'  Not 

KY  196 

Simulated _ 

Audio  Panel 

KING 

Audio  ChanneT 

NAV  &  Radio  Aids  Boards 

M  Beacons 

KMA  244 

Select.  I dent  Keying 

X-Talk  & 

Indicators _ 

Directional  Gyro 

KING 

For  HSI 

Aircraft  Wiring  Only 

KG  1  OCA 

DME  Receiver 

KING  ” 

For  DME 

Aircraft  Wiring  Only 

KR  63 

DME  Indicator 

KING 

DME  Ind 

DME  Serial  Data  (40  Bit 

KDI  57? 

Format) 

Automatic  Direction  Finder 

KING 

Selected  ADF  Station 

Instructor 

KR  87 

Input 

Marker  Beacon  Receiver 

KING 

Marker  Lamps 

3  Relays 

KMR  675 

Dual  Tachometer 

INSCO 

Rotor  i  Turbine 

2  Tacho  Drive  3oards 

Speed 

N1  (Ga^,  Producer) 

INSCO 

N1  Speed 

1  Tacho  Drive  Board 

Tachometer 

Fuel  Quantity 

INSCO 

Fuel  Qty 

1  Analogue  Output 

5707-3007 

-10  Volt,  dc 

Fuel  Pressur/Load 

Insco 

Fuel  Press 

2  Analogue  Outputs 

9016-J020 

Electric  Load 

1  jOP  Ampl if ier  _ 

Engine  Oil  Temp/Press 

INSCO 

Engine  Oil 

1  Analogue  Output 

9036-3023 

Temp /Press 

Digital  To  Resistance  Card 

Transmission  Oil  Temp/Press 

INsCO 

Transmlss ion  Oi 1 

1  Analogue  Output 

9036-3024  _ 

Temp /Press 

DiqiLal  To  Resistance  Card 

torque  Indicator 

INSCO 

%  Torque 

1  Analogue  Output 

4344-3048 

5  ♦  0  to  100  m  volt 

Turbine  Outlet  Temp 

INSCO 

TOT 

1  Analogue  Output 

502 1-3001 

Deq.  C. 

0  to  47  m  volt 

_  _ 

•  1GURE  t.  IRIAD  INSTRUMENTS 
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Navigation  System.  Figure  10  shows  the 
Navigation  System  when  configured  for  simulation. 
The  figure  shows  signals  that  simply  route 
through  the  breakout  panel,  some  that  originate 
in  the  simulation  computing  complex  and  some  that 
route  data  back  into  the  simulation  computing 
complex.  The  column  to  the  left  of  the  figure 
represents  the  simulation  computing  complex 
interface  electronics,  the  breakout  panel  is 
shown  centrally  and  the  aircraft  equipment  shown 
to  the  right. 

The  Horizontal  Situation  Indicator  (HSI)  is  the 
primary  navigation  instrument  and  independantly 
displays  heading,  course  and  glides  lope  deviation 
and  various  warning  flags.  Each  of  these 
indicators  have  a  separate  drive  signal  as  shown 
in  the  figure.  The  HSI  heading  repeater  signal 
to  the  Radio  Magnetic  Indicator  (RMI)  is  not 
broken  by  the  breakout  panel  and  is  used  as  the 
servo  feedback  to  the  HSI  heading  stepper  motor 
drive.  In  the  flight  configuration  the  complete 


drive  and  servo  control  is  done  by  the  Remote 
Directional  Gyro  KG  102A.  RMI  heading  is  slaved 
to  the  HSI  heading. 

The  Automatic  Direction  Finder  (ADF)  bearing  is 
driven  from  the  simulation  computing  complex 
interface  as  three  analogue  outputs  in  the  form 
of  dc  sin,  dc  cosine  and  reference  values.  The 
Vertical  Omni-directional  Radio  (VOR)  indicator 
is  driven  by  16  bit  serial  data,  routed  through 
the  breakout  pane1  and  generated  by  a  special 
card  in  the  simulation  computing  complex  inter¬ 
face.  The  Marker  Lamps  are  driven  by  6  volt 
power  relays  that  are  triggered  by  discrete  out¬ 
puts  from  the  simulation  computing  complex  inter¬ 
face. 

The  Distance  Measuring  Equipment  (DME)  is  driven 
from  the  simulation  computing  complex  interface 
by  a  special  card  that  emulates  the  aircraft  KR63 
DME  Receiver  and  provides  the  digital  40  bit  for¬ 
mat  data  necessary  to  drive  the  indicator. 
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Aircraft  Power.  The  aircraft  power  for  ail 
systems  is  derived  'from  a  +?8  Volt  DC  Power  Bus. 
Figure  11  shows  the  simulation  schematic  for  the 
28  volt  power  system.  The  simulation  computing 
complex  interrogates  the  aircraft  Battery  Switch 
and  Ground  Power  Switch,  and  when  either  are 
sensed  as  ON  the  power  relay  in  the  interface  is 
activated,  which  ;n  turn,  activates  the  external 
power  relay  in  the  aircraft.  All  aircraft  cir¬ 
cuit  breakers  are  routed  through  the  breakout 
panel  so  that  when  in  the  simulation  mode,  the 
condition  of  the  circuit  breakers  V0N  or  OFF)  is 
known  to  the  simulation  complex.  The  simulation 
software  determines  the  corresponding  condition 
of  the  system,/ indicator  and  produces  appropriate 
drive  output  to  the  aircraft. 

Flignt  Control  Position.  Five  positional 
potentiometers  have  been  installed  at  appropriate 
positions  on  the  pilot  control  linkages  to  pro¬ 
vide  positional  information  to  the  simulation 
computing  complex.  The  monitored  flight  controls 
are: 

(a)  Fore  and  Aft  Cyrlir 

(b)  Left  and  Right  Cyclic. 

(c)  Throttle 

(d)  Collective 

(o'1  Anti -Torque  Pedals 

The  following  figures  show  the  installation  of 
the  flight  control  potentiometers: 
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1  potentiometers  are  electrically 
the  aircraft  since  they  have  no 
flight  configuration.  Resolution 
Dotentiomete;  s  is  as  follows: 


R a nge  of  Tra v 
35  degrees 
30  degrees 
130  degrees 
6  centimeters 


Resolution 


10  arc  mins 


Safety.  Special  safeguards  are  required  when 
utilizing  real  aircraft  equipment  for  simulation 
purposes.  Each  breakout  panel  connection  is 
uniquely  keyed  so  that  there  is  no  possibility  of 
incorrect  connections  causing  inappropriate  elec¬ 
trical  power  routing.  Systems  that  should  not  be 
operated  when  the  aircraft  is  configured  for  sim¬ 
ulation,  i.e.,  pitot  heat,  engine  starter  and 
igniters,  are  disabled  by  appropriate  circuit 
design  of  the  breakout  panel  connections. 

Careful  design  is  incorporated  in  the  simulation 
computing  complex  interface  to  ensure  the  air¬ 
craft  units  will  not  be  subject  to  over  voltage 
conditions.  Power  is  not  applied  to  the  aircraft 
until  the  simulation  computer  is  active.  Addi¬ 
tionally  the  engine  starter  circuit  is  routed 
through  all  eight  breakout  panel  aircraft  config¬ 
uration  connections  so  that  the  aircraft  cannot 
be  operated  unless  every  connection  is  correctly 
in  place. 

An  Emergency  Stop  switch  is  located  near  the  left 
hand  seat  bulkhead  when  in  the  simulation  config¬ 


uration.  The  stop  switch  removes  all  electrical 
and  hydraulic  power  to  the  aircraft  when 
acJ ioned. 

Hydraulic  System.  Figure  16  shows  the 
helicopter  hydraulic  circuit  when  configured  for 
simulation.  An  auxiliary  power  unit  provides 
power  at  the  same  pressure  and  flow  as  that 
experienced  in  the  real  aircraft.  The  hydraulic 
connection  is  made  at  the  aircraft  "quick 
disconnect"  couplings  {units  10  in  the  figure). 
The  operation  of  the  auxiliary  power  unit  motor 
starter  and  solenoid  valve  are  controlled  from 
the  simulation  computer  complex  to  simulate  the 
effects  of  engine  speed  on  available  hydraulic 
pressure. 

The  aircraft  servo  actuators  provide  movement  to 
the  aircraft  control  devices  and  remove  feedback 
forces  to  the  pilots  hands.  By  powering  the  air¬ 
craft  hydrualic  system  when  in  the  simulation 
configuration,  the  "feel"  experienced  by  the 
trainee  pilot  will  be  representative  of  that  of 
the  real  aircraft  in  flight. 
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Simulator  Computing  Complex 

Computer  and  Peripherals.  The  TRIAD  compu¬ 
tational  sytem  uses  the  SYSTEMS  32/2750  General 
Purpose  Computer  and  includes  a  disc  based  oper¬ 
ating  system  suitable  fcr  simulation  applica¬ 
tions  . 

The  computer  complex  schematic  is  shown  in  Figure 
17  and  consists  of  one  80  mega  byte,  moving  head 
disc  drive;  one  magnetic  tape  unit  for  develop¬ 
ment  support  and  data  backup;  one  175  CPS  print¬ 
er;  two  CRT  editor  terminals,  two  CRT  Digital 
Readout  Unit  &  Interactive  Displays  (DRUID);  and 
one  CRT  Operator's  Console  (OPCON).  The  OPCON 
replaces  the  mechanical  keyboard  and  greatly 
enhances  operation  and  maintenance  of  the  compu¬ 


ter  complex.  The  DRUID  enables  direct  memory 
access  for  on-line  alteration  or  debug  of  simula¬ 
tor  software  systems. 

The  computer  configuration  for  TRIAD  consists  of 
a  single  slot  CPU,  512KB  interleveled  error  cor¬ 
recting  memory,  an  MTU  processor,  a  disc  proces¬ 
sor,  two  I/O  devices  (High  Speed  Device  Inter¬ 
face)  capable  of  3.2  megabytes  bi-directional 
data  transfer,  and  an  Input/Output  Processor 
which  handles  multiplexing  between  the  main  CPU 
data  bus  and  a  multipurpose  bus  used  for  external 
devices  such  as  the  OPCON.  The  real  time  clock 
signal  is  provided  by  the  linkage  and  all  hard¬ 
ware/software  is  slaved  to  this  one  timer  to 
provide  exact  syncronization  of  all  simulated 
systems . 


Computer  Software.  The  software  used  for 
TRIAD  can  be  separated  into  three  major  sections: 
1)  Operating  Systems;  2)  Simulation  modules  and 
their  executives;  3)  Simulator  support  programs 
including  maintenance,  t^st  and  diagnostics.  A 
maximum  iteration  rate  of  30  Hz  is  used  to  en¬ 
sure  as  high  a  fidelity  as  possible  in  the  per¬ 
formance  areas  such  as  flight  dynamics,  flight 
control  interface  and  visual  transfer.  Other 
simulator  systems  modules  are  installed  at  lower 
rates  but  in  all  cases  at  a  rate  which  provides 
adequate  time  for  accurate  duplication  of  that 
system.  Figure  18  shows  the  family  tree  of  soft¬ 
ware  units  that  make  up  TRIAD  computer  software. 

Operating  System.  The  operating  system  soft¬ 
ware is~an^edrffusTbn  modified  version  of  the 
SYSTEMS  MPX-32  Disc  operating  System.  These  mod¬ 


ifications  include  software  which  shorten  module 
computation  time  by  placing  math  subroutines  in 
memory  to  be  called  as  needed,  and  enables  real 
time  memory  access  through  DRUID.  The  math  sub¬ 
routines  enable  rapid  calculations  of  trigono¬ 
metric  functions,  integration,  interpolation, 
function  generation,  etc.  thus  shortening  CPU 
execution  time  for  simulator  modules.  DRUID 
allows  memory  access  through  CRT/keyboard  manip¬ 
ulation  a r.d  enables  the  user  to  address  any 
location  in  the  simulator  module  and  the  state  of 
all  CPU  registers  at  each  location.  Entries  can 
be  made  in  hexi decimal,  decimal  or  character 
codes.  Indexed  instructions  can  be  monitored  at 
the  exact  index  register  specified  by  the  code. 
Any  changed  memory  location  will  remain  in  that 
altered  state  until  changed  again  by  DRUID  or 
until  a  new  system  is  loaded  from  disc. 
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SimulatiO;.  Modules  and  Their  Executives. 
These  programs' '"TipTerient  information  processing 
to  simulate  aircraft  performance,  on-board  sytems 
peformance,  aircraft  c-ound  environment,  weather 
and  atmosphere,  navigation  and  radio  aids  systems 
and  instructor  functions  including  accurate  de¬ 
piction  of  malfunctioning  systems  as  selected  by 
the  instructor.  The  accurate  dynamics  of  systems 
such  as  flight,  fuel,  electrical,  engines, 
sounds,  radio  navigation,  etc.,  are  present  in 
the  TRIAD  simulator  load.  The  Master  Executive 
module  provides  the  order  and  time  of  execution 
of  each  sub  system  module,  and  maintains  priority 
in  the  CPU  foreground  tasks  to  insure  all  simula¬ 
tor  tasks  are  accomplished  as  called.  Background 
tasks  (i.e.  editing,  cataloging,  Library  Editing, 
Batch  jobs)  may  be  done  as  desired  even  in  the 
normal  simulator  training  mode  if  operation. 

Simulator  Support  Programs  Tbe  Simulator 
Support  Programs,  a  few  of  which  were  ir.entioned 
above,  allow  the  simulator  operator  to  manipu¬ 
late,  edit,  modify,  assemble,  cempile  and  run  any 
program  in  the  simulator  compiter.  New  source 
programs  may  be  added,  if  desired  or  existing 
ones  modified  to  fulfill  the  nieds  of  the  opera¬ 
tor.  A  Ground  Station  Data  :,GSD)  Compiler  and 
Editor  is  included  to  ena^e  programming  of  vir¬ 
tually  any  navigation  aid  in  the  world  for  use  by 
the  simulator  aircrew  for  training.  375  stations 
may  be  active  at  ^ny  one  time,  with  the  maximum 
number  of  stations  available  for  call  limited 
only  by  disc  storage  area.  S.E.L.  Level  I  &  II 
software  diagnostics  are  included  as  a  mainten¬ 
ance  aid.  The  simulator  load  can  not  be  run 
while  level  I  or  II  is  being  executed  as  it 
demands  full  time  use  of  the  CPU,  the  SELBUS,  and 
any  peripherals  attached.  These  diagnostics 
isolate  malfunctions  to  a  major  assembly  (circuit 
Curd)  level,  thereby  minimizing  down  time  with 
board  replacement  being  acomplished  by  the  user. 

The  majority  of  source  programs  for  TRIAD  are 
written  in  SYSTEMS  assembler  language  to  minim¬ 
ize  computational  time.  However,  many  support 
and  control  programs  are  written  in  FORTRAN. 

Simulator  Linkage  Interface.  TRIAD  utilize* 
an  interface  system  which  tal.es  maximum  advantage 
of  proven  state-of-the-art  techniques  in  elec¬ 
tronic  component  technology,  packaging  and  data 
transmission.  The  operational  characteristics  of 
the  interface  are  matched  to  the  high  speed  data 
transmission  capabilities  of  the  SYSTEMS  32 
series  computer  and  the  almost  exclusive  use  of 
aircraft  instruments  throughout  the  flight  deck. 
The  interface  is  fully  integrated  with  the  sound/ 
vibration  system  and  is  capable  of  meeting  the 
high  speed  response  demands  of  helicopter  flight 
and  instrument  systems. 

Figure  19  shows  the  disposition  of  individual 
modular  units  comprising  the  interlace  system. Key 
features  in  the  design  of  the  interface  include:- 


Function  Boards.  Aircraft  instrument  drive 
signals  are  derived  from  a  selection  of  standard 
function  board  channels  that  removes  the  need  for 
complex  special -to-aircraft  type  system  beards. 

Power  Supplies.  The  interface  cabinet  is 
equipped  with  the  required  power  supplies  that 
are  fully  protected  by  circuit  breakers. 

Cockpit  Power  Supply.  In  addition  to  -he 
ac/dc  power  required  for  the  linkage  electronics, 
an  external  source  of  regulated  28  VDC  power  is 
necessary  for  the  cockpit  environment.  A  30  amp 
power  unit  is  located  in  a  separate  cabinet  and 
provides  power  to  both  the  linkage  and  cockpit. 

Diagnostics .  A  comprehensive  range  of  inter¬ 
face  diagnostics  operate  on-line,  during  train¬ 
ing.  Each  interface  channel  is  interrogated  at 
regular  intervals  and  failures  are  identified  on 
the  operator  output  device.  This  enables  the 
defective  unit  to  be  remrvni  and  replaced. 

Interface  Cabinet  Layout 

The  cabinet  layout  for  TRAID  contains  five  levels 
of  equipment,  where  each  level  is  178  mn.  (7  in) 
high,  as  shown  in  Figure  19. 
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Multiplexed  Digital  Data  Transmission.  The  ▼ 

primary  dala  bus  (PDB)  transmits  multiplexed  [  DENOTES  BUS  INTERFACE  BOARD 

digital  data  in  two  cables  between  -he  simulator 
computer  and  the  interface  cabinets. 
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The  top  level  contains  the  cabinet  facilities 
unit.  The  unit  distributes  the  various  power 
lines  and  services  f .  om  the  power  distribution 
bay  to  the  ac  and  dc  power  supplies  within  each 
card  cage.  Smoke  and  overheat  detectors  and  fan 
units  are  also  contained  within  the  facilities 
unit. 

The  central  four  levels  contain  the  interface 
electronics  designed  to  accommodate  a  range  of 
standard  cards  capable  of  driving  instruments  and 
indicators  normally  used  in  aircraft.  The  range 
of  cards  used  for  TRIAD  are  listed  below: 


Analogue  Input  Analogue  Output 

Discrete  Tnput  Discrete  Output 

Addressable  Relay  Synchro  Driver 

Servo  Amplifier  QUAD  Amplifier  Card 

Power  Amplifier  Sound  System  Cards 

Nav.  a/id  Radio  Aids  System  Cards. 

These  cards  are  arranged  as  equipment  modules  as 
shown  in  Figure  19. 

Coupler  Module.  The  coupler  module,  controls 
all  communication  between  the  SYSTEMS  32  host 
computer  and  the  various  modules. 

The  SYSTEMS  32  computer  contains  a  high  speed 
direct  memory  access  channel  known  as  the  high 
speed  data  interface  (HSDI).  This  connects  to 
the  coupler  module  via  a  bi-directional  highway 

known  as  the  primary  data  bus  (PDB).  A  unidirec¬ 
tional  data  highway  originates  at  the  coupler  and 
loops  around  the  on-board  equipment  before 
returning  Lo  the  coupler.  Each  of  the  equipment 
modules  connect  to  the  bus.  Messages  originating 
from  the  coupler  module  are  transmitted  around 
the  bus  and  carry  data  to  and  from  the  various 
modules . 

Panel  Drive  Module.  This  module  contains  the 
electronics  associated  with  aircraft  panels  and 
consists  predominantly  of  discrete  inputs  boards 
(rrunitoring  switch  positions)  and  discrete 
outputs  (driving  lamps). 

Instrument  Drive  Module.  This  module  con¬ 
tains  tFe  electronics  associated  with  aircraft 
instruments,  i.e,  synchro  drives,  servo  ampli¬ 
fiers,  various  k^INC  serial  transmi .iers,  etc. 

So  md  and  Rag;  a  Aids  Module.  Level  5  of  the 
linka"ge  contains  TRe  elec  Ironies  necessary  to 
generate  206B  sounds  including  environment  out- 
c 1 *.K  ‘'ockoit.  Also  ii.  this  module  are  the 
electronics  necessary  to  dr’ve  the  navigation 
instruments . 

Sound  Simulation 

After  an  extensive  analysis  of  the  sounds  pro¬ 
duced  by  a  206B  helicopter  in  all  modes  of  oper¬ 
ation,  the  following  innovative  techniques  were 


employed  to  surround  the  pilot  with  a  realistic 
atmosphere  of  saund: 

i  oint  sources  o'  individual  noises  were  isolated 
to  the  origin  from  which  they  were  generated. 

Pressure  levels  ard  volume  of  noise  were  measured 
from  each  source. 

Unique  sounds  (i.e.  "blade  slap",  blade  stall, 
skid  slide  noise)  were  isolated  and  the  frequen¬ 
cies  of  each  were  identified. 

Interior  sounds  such  as  boost  pumps,  fuel  valves 
and  generator  sounds  were  recorded  and  identi¬ 
fied  . 

Blade  passing  sounds  and  frequencies  were 
obtained. 

To  more  accurately  simulate  this  aircraft,  real¬ 
istic  in  flight  sound  pressure  1  e  el s  are  pro¬ 
duced  to  bombard  the  interior  and  exterior  of  the 
cockpit  to  stimulate  the  sense  of  motion  and 
vibration  achieved  in  actual  flig.it.  It  was 
necessary  to  generate  110-114  dbA  in  order  to 
penetrate  the  cockpit  shell  with  enough  energy 
and  sound  realism  to  match  the  93-98  dbA  which 
normally  is  present  in  this  aircraft.  An  auxil¬ 
iary  sound  cabinet  receives  software  controlled, 
hardware  generated  composite  audio.  This  audio 
is  then  channelled  to  its  appropriate  point 
source  and  is  amplified  to  the  representative 
level  of  energy.  Very  high  efficiency  profes¬ 
sional  sound  equipment,  including  1/3  active 
equalizers,  frequency  crossovers,  power  amplif¬ 
iers  and  speakers,  are  employed  to  recreate  this 
envelope.  Point  source  sounds  are  blended  and 
matched  to  five  speaker  locations.  The  front 
left,  and  front  right  elevated  speakers  are  the 
origin  for  the  blade  swish  sound  and  accompaning 
blade  passing.  An  aerodynamic  hiss  is  also 
blended  here  to  provide  the  air  rush  *oise  heard 
over  the  canopy  during  flight.  A  three  speaker 
group  inside  the  rear  of  the  cockpit  centralizes 
sounds  which  originate  inside  the  ship.  Engine 
and  transmission  noises  are  the  primary  sounds 
heard,  but  also  blended  into  this  audio  are  boost 
pump  and  generator  noise.  Because  engine  and 
transmission  sounds  can  be  heard  outside  the 
cockpit,  there  are  two  additional  stacks  of 
speakers,  with  a  frequency  range  of  20-20KHZ, 
directly  on  the  left  and  right  sides  of  the  cock¬ 
pit  and  positioned  to  penetrate  the  shell  at  the 
same  angle  as  that  experienced  in  the  aircraft. 
Retreating  blade  sounds  are  blended  into  the  left 
outside  stack.  Blade  Slap  is  created  by  the 
advancing  blade  during  rapidly  changing  aero¬ 
dynamic  conditions  or  under  high  blade  loads  and 
is  reproduced  in  the  right  outside  stack.  All 
outside  speakers  are  modulated  with  a  13.15  HZ 
blade  passing  frequency  and  this  modulation 
coupled  with  a  low  frequency  (Below  30  HZ)  noise 
creates  the  thump  or  vibration  of  a  two  bladed 
helicopter.  Sounds  from  engine  start,  through 
hover  and  translational  lift  to  forward,  rearward 
or  sideward  flight,  into  cruise  and  even  to 
engine  out  autorotat ion  to  a  landiny  are  accur¬ 
ately  reproduced  to  provide  the  pilot  a  realistic 
environment. 
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Instructor  Control 


A  flight  simulation  cor..plex  usually  incorporates 
a  permanent  instructor  facility  located  either 
inside  the  simulator  cab  or  offboard  in  a  sepa¬ 
rate  room.  Because  TRIAD  employs  a  real  aircraft 
fuselage,  the  instructor  may  be  located  at  the 
co-pilot  station.  A  portable  instructor  facility 
is  required  that  has  the  capability  to  fully 
control  the  simulator  and  provide  malfunction 
conditions  and,  of  course,  be  easy  to  use. 

Vupoint  Portable  Display  Unit  Vupoint 
employs  a  touch  sensitive  plasma  display  driven 
by  a  National  Semiconductor  280  based  micropro¬ 
cessor.  Tf?  display  can  present  up  to  twelve 
lines  of  40  characters  each  per  page,  and  has 
internal  storage  capacity  of  up  to  64  pages. 
Figure  20  shows  the  Vupoint  display  unit.  The 
front  screen  assembly  can  be  detached  from  the 
main  casing  for  use  on  the  instructor's  knee  as 
shown  in  Figure  1. 

The  pages  are  arranged  in  a  logical  sequence  and 
include  an  index,  set-up  pages,  i.e.  weight,  CG 
etc,  resets,  freezes  and  system  malfunctions. 

To  select  a  page,  change  a  parameter,  or  insert  a 
malfunction,  the  instructor  simply  "touches"  the 
appropriate  text  on  the  page.  Infrared  transmit¬ 
ters  and  receivers  located  around  the  edge  of  the 
display  detect  the  touched  screen  position  that 
cause  system  software  to  implement  the  appropri¬ 
ate  action. 

Visual  System 

The  visual  system  consists  of  three  major  compon¬ 
ents:  The  computer  generated  image  system,  the 
calligraphic  projection  system  and  the  backscreen 
assembly. 

Computer  Generated  Image  System  The  image 
generator  is  a  nOvOV^EW  Day/Dusk/Night  image 
generation  (IG)  system  It  employs  a  TI-980B 
based  computer  processor  and  a  frame  rate  of  46 
HZ  during  daylight  mode  and  39  HZ  during  ausk  and 
nignt  operation.  The  IG  is  capable  of  up  to  four 
channels,  with  three  installed  and  operational  or. 
TRIAD.  The  IG  is  interfaced  to  the  simulator 
cor.puter  via  a  high  speed  DMA  interface.  Control 
and  pilot  eye  coordinate  and  attitude  data  are 
transferred  to  the  IG  each  simulator  computer 
f  rame . 

SP3T  is  a  follow-on  product  of  the  SP3  FAA  Phase 
III  certified  visual  system,  and  provides  all  the 
features  of  SP 3  plus  texture.  Various  textural 
patterns  can  be  applied  to  surfaces  in  the  scene 
to  give  the  appearance  of  clouds,  sea  waves, 
grass,  concrete,  forest,  etc.  This  image  genera¬ 
tion  system  is  incorporated  in  TRIAD  because  it 
is  believed  that  the  texture  feature  will  provide 
the  helicopter  trainee  pilot  the  precise  visual 
information  he  needs  to  successfully  fly  and  land 
a  helicopter. 

Cal  1 i graphic  Projection  System  The 
R  ed  if  fusion  C  aTT  1  graphic  Projector  can  d  i  sp  1  ay 
both  calligraphic  'ightpoints  and  raster  gene i - 


FIGURE  20  VUPOINT  CONTROL  UNIT 


ated  surface  during  a  single  IG  frame.  Three 
projectors  are  used  for  TRIAD  and  each  consist  of 
a  head  unit,  a  power  unit  and  a  remote  control 
unit. 

Projector  Head  Unit.  The  head  unit  is  an 
assembly  incorporating  three  projection  tube 
assemblies  (red,  green  and  blue)  and  their  assoc¬ 
iated  scanning  systems,  the  EHT  generator  and 
video  processing  equipment.  The  tube  assemblies 
are  housed  within  a  rigid  light-weight  aluminum 
casing.  Each  projector  head  unit  is  supported  cn 
a  frame  and  positioned  relative  to  the  corres¬ 
ponding  backscreen  for  correct  geometric  align¬ 
ment  , 

Projector  Power  Unit.  The  power  unit  is  a 
floor  standing  cabinet  containing  the  deflection 
power  supply,  -24  volt  power  supplies,  tube 
heater  supplies  and  video  and  HT  power  supplies. 

Projector  People  Control  Unit,  The  remote 
control unit  enables  the  projector  to  be  cal  lib- 
rated  from  the  pilot  viewing  position  inside  the 
helicopter  by  a  single  operator. 

Backscreen  Assembly  Each  projector  is 
mounted  behind  TV  display  screen  assembly  com¬ 
pletely  out  of  the  pilot's  view.  The  backscreen 
assembly  is  comprised  of  a  framework  supporting 
three  eight  foot  hiqh  by  six  foot  wide  flat 
translucent  panels.  The  panel  material  has  an 
optical  diffusing  coating  for  high  resolution 
images. 

Each  panel  is  oriented  such  that  the  pilot  views 
screen  at  a  perpendicular  angle  to  present  a  con¬ 
tinuous  horizontal  image.  The  size  of  this  image 
is  48°  vertically  and  108°  hor izonta  1  ly . 

Data  Bases  Several  data  bases  are  used  for 
TRIAD,  ranging  from  a  gulf  oil  rig,  a  ship  land¬ 
ing  platform,  to  a  standard  airfield.  Obstacles 
are  located  at  c»  itical  locations  in  the  data 
bases  to  ensure  pilot  skills  are  exercised  to 
their  maximum  during  TRAID  pilot  assessment. 
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RESULTS 

The  objective  of  adapting  a  certified  helicopter 
for  use  as  a  full  high  fidelity  simulator  to  be 
used  for  pilot  training  was  achieved. 

"Embedded  Simulation"  Concept 

All  of  the  objectives  to  prove  the  "Embedded  Sim¬ 
ulation"  concept  were  achieved.  The  generic  hel¬ 
icopter  flight  simulation  model  was  successfully 
modified  to  approximate  the  Bell  2068  performance 
characteristics.  All  of  the  aircraft  systems 
w*re  utilized  in  conjunction  with  the  simulation 
complex  and  the  out-the-window  CIG  Visua1  Display 
System. 

Total  Simulation  System  Validation 

Four  visual  models  were  used  to  evaluate  sytem 
capability  and  pilot  performance.  These  models 
were: 

1)  Montgomery  County  Airport,  Conroe,  Texas 

2)  Perry  Class  Frigate  Underway 

3)  Offshore  Oil  Drilling  Platform 

4)  Nuremburg,  Germany,  Municipal  Airport. 

’ lode  1  1  provided  the  capability  to  demonstrate 
hovering,  air  taxiing,  pedal  turns,  transition 
to  forward  flight,  climbs,  decents.  -pp**oach  to 
landing  and  landings.  V  i s u i 1  cues  and  system 
response  were  adequate  provide  a  training 
capability  for  notice  pilots  as  well  as  profic¬ 
iency  trai°  :.^  tor  the  experienced  pilot. 

Model  2  provided  a  helipad  on  a  frigate  underway 
at  sea.  Cues  such  as  a  relating  radar  antenna, 
bow  wave  and  wake,  and  moving  reastate  were  pro¬ 
vided  allowing  the  pilot  to  practice  normally 
difficult  landings. 

Model  3  demonstrated  the  realism  of  operating  on 
an  offshore  oil  drilling  platform  providing  the 
capability  to  enhance  pilots  training  for  all 
levels  of  qualification. 

Model  4  provided  a  full  IL3  approach  capability 
as  well  as  the  normal  runway  and  traff  pattern 
functions.  Category  II  approaches  to  minimums 
were  utilized  as  well  as  simulated  emergencies 
such  as  transmission  failures  and  chip  lights. 
The  full  rarge  of  autorotation  practice  as  a 
result  of  engine  failures  was  also  demonst  ued. 
Hovering,  straight  in,  and  180°  autorotat ions  to 
full  stop  landings  were  accomplished  with  a 
degree  of  realism  that  provided  the  touchdown 
response  appropriate  to  the  degree  of  impact. 

Computer  Image  Generated  V  isual  Display 

The  wide  screen  with  rear  projection  provided  a 
real  f.on-col  1  imated  image  with  a  108  deqree  hori¬ 
zontal  and  48  degree  vertical  f ield-of-view.  This 
view  is  continuous  and  can  be  viewed  from  any 
position  in  the  cockpit  without  losing  or  having 
the  visual  fade.  The  texturing  of  the  visual 
display  provided  the  critical  cues  for  heiqht, 
closure  and  acceleration  that  are  mandatory  for 
low  level  helicopter  flyir^.  The  realism  lines 


on  the  concrete  runways  and  taxiways,  the  tire- 
marks  or.  the  runways,  the  mottled  ocean  surface, 
grass  and  tree  tops  and  the  overcasts  and  cloud 
surfaces  are  all  authentic  and  lend  to  the  real¬ 
ism  of  the  simulation. 

Sound  and  Vibration 

The  sound  system  provided  a  high  degree  of  real¬ 
ism  by  including  actual  engine,  transmission, 
rotor,  slipstream  and  touchdown  vibrations  of  the 
operating  aircraft  to  the  pilot. 

Simulation/Aircraft  Interface 

The  goal  of  adapting  a  certified  helicopter  for 
simulation  use  and  return  to  full  operation  in 
two  hours  or  less  was  more  than  met.  It  was 
demonstrated  that  the  Bell  206B  could  be  placed 
in  operation  as  a  simulator  in  ^4  minutes  and 
returned  to  full  flight  operation  in  15  minutes 
by  two  people.  Figure  21  show  the  TRIAD  heli¬ 
copter  during  its  proving  flight. 


FIGURE  21  TRIAD  PROVING  FLIGHT 


General  Results/Comments 

The  TRIAD  concept  was  demonstrated  as  a  produc¬ 
tion  prototype  for  three  months  prior  to  this 
paper  submission.  The  objective  was  to  expose 
the  concept  to  a  broad  range  of  aviation  related 
personnel  and  to  gather  comments  on  the  prototype 
system. 

All  participants  in  the  d^monstrat ions  were 
advised  of  the  prototype  status  and  the  various 
areas  which  would  be  either  improved  or  modified 
in  the  production  systems.  Each  participant  was 
encouraged  to  critically  evaluate  lie  system  on 
the  basis  ot  both  inputs.  The  production  time 
scale  would  allow  for  inclusion  of  almost  any 
modification  that  surfaced  consistently  as  a 
result  of  these  demonstrations. 

Upon  termination  of  the  demonstration  phase,  the 
only  consistent  result  was  that  the  sy^em  func¬ 
tioned  remarkably  well  as  a  trainee  device. 
Pilot  acceptability  of  the  flight  characteris¬ 
tics,  as  compared  to  a  Bell  206  aircraft,  was 
extremely  positive. 


The  absence  of  a  motion  base  on  the  system  was 
emphasized  in  each  pre-brief  to  participants  as 
being  one  of  the  major  departures  from  conven¬ 
tional  simulation  devices.  Each  participant  was 
asked  to  critically  evaluate  this  particular 
aspect  during  the  demonstration.  The  overwhelm¬ 
ing  concensus  was  that  the  absence  of  motion  was 
not  particularly  noticed  given  the  aoparent 
motion  created  by  the  visual-sound  combination. 
Further,  concensus  indicated  that  pilots  who  had 
received  training  on  current  motion  based  heli¬ 
copter  simulators  were  able  to  achieve  a  more 
rapid  accommodation  to  the  system  and  performed 
low  level  flight  and  hover  maneuvers  with  greater 
ease.  The  ability  to  operate  effectively  at  very 
low  altitudes  with  a  TRIAD  system  may  resolve  the 
question  of  motion  versus  non-motion  in  flight 
simulators. 


While  the  results  gained  from  the  demonstration 
phase  are  not  presumed  to  be  scientific,  the 
overwhelming  positive  response  to  the  TRIAD  sytem 
would  indicate  that  the  approach  is  valid  and 
that  acceptability  as  a  training  device  is  fairly 
unanimous. 


Cost  Analysis 

For  a  given  visual  system,  TRIAD  will  represent  a 
considerably  smaller  initial  investment.  For  pur¬ 
poses  of  example  only,  assume  an  acquisition  cost 
of  approximately  S4.5M  for  TRIAD  (tc  include  one 
software  package);  S750K  for  additional  TRIAD 
aircraft  interface  software  packages;  and,  S6.0M 
for  a  conventional  simulator.  These  assumptions 
achieve  a  savings  of  approximately  25  percent  for 
a  one  aircraft  system. 

Using  the  above  cost  approximations,  assume  a 
situation  in  which  five  aircraft  types  can  be 
simulated.  The  TRIAD  system  cost  would  be  S4.5M 
+  $750*.  x  4  =  S7.5M.  The  conventional  simulator 
would  cost  S6.0M  x  5  =  $30. OM.  These  assumptions 
achieve  a  savings  of  approximately  75  percent  for 
a  five  aircraft  system. 

TRIAD  will  represent  the  same,  or  better,  results 
in  simulator  versus  ai-craft  training  trade-offs 
when  compared  to  a  conventional  simulator. 

A  study  by  Orlansky  and  String  (1979)  gives  con¬ 
siderable  insight  in  the  various  areas  of  simula¬ 
tion  cost  analysis  that  should  be  considered. 


FUTURE  DEVELOPMENTS 


The  main  objective  of  TRIAD  was  to  prove  that 
Embedded  Simulation  is  a  concept  that  has  prac¬ 
tical  application  for  pilot  training.  The  most 
expedient  technical  approach  was  employed  on  the 
basis  that  a  nore  general  approach  would  be 
implemented  sroulu  TRIAD  bn  successful.  The 
lessons  learned  during  this  project  lead  to  the 
following  future  developments  that  will  provide  a 
better,  more  flexible  total  simulation  system. 


Pneumatic  Instrument  Drive  System 

TRIAD  was  configured  with  ;  .  Air  Data  Computer 
instrument  system  permitting  electrical  drive  tc 
the  Altimeter,  Airspeed  Indicator  and  Vertical 
Speed  Indicator.  A  planned  TRIAD  development  is 
to  apply  pneumatic  pressure/vacuum  directly  to 
the  pitot  head  and  static  source  sensors  in  a 
precise  manner  to  drive  the  air  driven  instru¬ 
ments  tc  a  resolution  comparable  to  that  exper¬ 
ienced  in  flight. 

Dedicated  Simulation  Model. 

TRIAD  employed  generic  flight,  rotor  and  engine 
simulation  models  that  were  suitably  adjusted  tc 
provide  approximate  Bell  206  performance.  Future 
TRIAD  products  will  employ  simulation  models  for 
the  aircraft  systems  that  have  been  rigorously 
designed  from  aircraft  manuf acturers  substan¬ 
tiated  data. 

Cont -ol  Loading 

Many  small  helicopters  do  not  errploy  fully 
boosted  hydraulic  control  units  on  all  its  flight 
control  systems.  The  anti-torque  system  on 
TRIAD,  for  example,  was  not  boosted  and  therefore 
does  not  provide  the  aerodynamic  feedback  forces 
normally  experienced  by  the  pilot. 

It  is  planned  to  develop  a  small,  highly  portable 
control  loading  unit  that  can  easily  be  mounted 
to  any  aircraft  frame  and  control  linkage,  that 
can  readily  be  calibrated  and  will  provide  appro¬ 
priate  force  feedback  to  the  pilot  under  all 
normal  maneuvering  conditions. 

Visual  Display  Improvements 

The  non  symmetrical  f ield-of-v iew  proved  not  to 
be  ideal  since  helicopter  maneuvers  usually 
require  equal  vision  on  either  side  of  the  cock¬ 
pit.  The  helicopter  will  be  positioned  with  the 
three  display  units  placed  symmetrically  about 
the  pilot  producing  a  48  degree  vertical  by  108 
degree  horizontal  f ield-of-view.  It  is  also 
planned  to  jssess  a  four  screen  144  degree  hori¬ 
zontal  f iel J-of-view  display  configuration. 

Vibration 

The  pilot  and  copilot  seats  on  the  Bell  206  fit 
over  a  spacious  part  of  the  helicopter  fuselage 
containing  the  flight  control  linkages.  To  aug¬ 
ment  the  sound/vibration  system,  it  is  planned  to 
design  a  “seat  vibrator"  that  can  be  located  ir 
this  space  under  each  seat.  It  is  important  that 
vibration  is  r.ot  induced  in  the  aircraft  frame¬ 
work,  engine  or  transmission  and  therefore  a  seat 
vibrator  employing  seismic  principles  is  being 
considered. 
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F-16  &  A-1UA  OFT  SIMULATOR 
FLIGHT  SYSTEMS  DEVELOPMENT  &  TEST 
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ABSTRACT 
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'""This  paper  discusses  flight  systems  development  and  test  issues  of  the  Air  Force's  F-16 
and  A- 1 0  Operational  Flight  Tra.ners  (OFT)  for  flight  controls,  performance,  and  stability 
and  conlrot.  Brief  descriptions  of  the  aircraft,  simulators  and  their  hardware,  and  flight 
systems  software  are  presented.  The  basic  design  data  base  is  described;  simulator  test 
techniques  are  presented;  and  some  of  the  more  interesting  flight  system  simulation  test 
problems  and  their  resolutions  a>  discussed.  Probably  the  more  basic  reason  for  many  of 
the  A- 1 0  OFT  flight  systems  initial  problems  relates  to  the  minimal  involvement  of  user 
pilots  during  data  development  and  flight  system  logic  design  operation.  Conversely ,/t.he 
success  of  initial  government  flight  systems  testing  of  the  F-16  OFT  was  aided  by  early'- 
user  pilot  involvement  in  system  design  and  ooeration.  This  paper  also  contends  that  the 
greatest  amount  of  transfer  of  training  (simulator  to  aircraft)  for  flight  systems  opera¬ 
tion  and  performance  is  obtained  with  a  design  philosophy  which  replicates  cockpit  features, 
visual  cues,  and  the  performance  of  the  actual  aircraft.  The  paper  concludes  with  suggested 
future  methods  to  improve  simulation  performance  and  test  efficiency 


NOMENCLATURE 

AOA 

angle  of  attack 

CLSe 

lift  coefficient  due  to  elevator 

DAC 

digital  to  analog  converter 

DPS 

degrees  per  second 

FS/g,  Se/g 

stick  force  and  elevator  pet- 
load  factor 

H5T,  rWT 

high  speed  tunnel,  polysonic 
wind  tunnel 

I/O 

input/output 

s/w 

software 

VVI 

vertical  velocity  indicator 

AjKf  UAj; 

The  F-16A  is  a  single-engine,  single  seat  mul¬ 
tirole  tactical  fighter  with  full  air-to-air  and 
air-to-ground  combat  capabilities.  It  has  a  wing¬ 
span  of  33  feet  with  wing  tip  missiles  and  an 
overall  length  of  49.5  feet.  With  full  internal 
fuel,  full  ammunition  and  two  AIM-9  missiles,  the 
gross  weight  of  the  aircraft  is  approx imately 
23,500  pounds.  The  maximum  gross  landing  weight 
is  27,500  pounds  and  the  maximum  gross  takeoff 
weight  is  35,000  pounds,  which  permits  the  car¬ 
riage  of  almost  11,000  pounds  of  externa!  stores. 
The  F-16A  aircraft  is  pown^cd  by  the  F100-PW-1C0 
turbofan  engine,  which  is  ir.  the  26,000  pound 
thrust  class.  The  fuselage  is  charac teri zed  by  a 
large  bubble  canopy  and  an  underslung  engine  air 
inlet.  The  wing  tail  surfaces  are  thin  and  fea¬ 
ture  moderate  aft  sweep  (40°).  The  wing  is  a  NACA 
64A204  airfoil  wnich  has  leading  edge  flaps  that 
are  deflected  automat ical  ly  to  enhar.ee  performance 
over  a  wide  speed  range.  Flaperons  art-  mounted  on 
the  trailing  edge  of  the  wing  and  combine  the 
functions  of  flaps  and  ailerons.  The  hori cental 
stabilizers  have  a  small  amount  of  anhedral  (30°) 
and  provide  pitch  and  roll  control  through  differ¬ 
ential  deflection.  The  vertical  tail,  augmented 
by  twin  ventral  fins,  provides  directional  stabil¬ 
ity.  The  primary  flight  confol  system  is  a  full 
fly-by-wire  system  which  does  not  use  mechanic.:' 
linkages  or  control  cables  between  the  cockpit  any 


the  control  surfaces.  This  systems  provides  three- 
axis  flight  path  control  through  the  use  of  a  side 
stick  controller  and  rudder  pedals. 


Fig  !  F-16A  Aircraft 


The  F-16  Operational  Flight  Trainer  (OFT)  i 
currently  nearing  the  end  of  a  four-year  production 
p?onram  and  is  being  installed  at  various  training 
sites  throughout  the  United  States  Europe,  and 
A  ia.  The  F-16  OFT  is  designed  and  built  by  the 
Singer-Link  Corporation  in  Binghamton,  New  York 
and  is  intended  to  provide  pilots  wit',  flight 
training  that  is  directly  tr ansferrahle  to  thef-16 
aircra4'.  Through  the  use  of  the  trainer,  experi¬ 
ence  can  be  gained  in  the  operational  use  of  all 
aircraft  systems.  Pilots  are  able  to  practice 
tactical  missions,  both  air-to-air  and  air- to- 
ground,  with  all  possible  weapon  loadings.  Emer¬ 
gency  procedures  can  safely  be  practiced  and  the 
trainer  can  be  ‘’flown"  to  the  limits  of  the  air¬ 
craft's  flight  envelope  to  provide  increased  con¬ 
fidence  -id  survi vabi  1  i ty . 

•!-.*! Nf • 

C omputation  system .  The  current  F-16  OFT 
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consists  of  a  computational  system  which  compr’s  s 
a  complex  of  hardware  units  and  structured  SOft- 
warp  iirnnranK  (Fin  ?)  Thp  ma  i  n  .  nmnijtpr  rr.n*i<;ts 

of  a  central  Nord-10/s  16  bit  processor  expanded 
four  32  bit  Nord-60  processors.  The  Ncrd-10/s 
provides  central  control  and  supervision  for  all 
system  I/O  and  program  execution.  The  Nord-50 
processors  are  dedicated  to  simulation  system  pro¬ 
cessing  under  control  of  the  central  Nord-10/s. 

The  Nord-50  processors  run  in  pa-alle1 2 3 4 5 6 * 8 9 10,  each  con¬ 
tributing  an  uninterruptible  one-fourth  of  the 
to t d 1  Nord-!0/5C  computing  power.  Additionally, 

twij  H  nt -dev?1  Oned  linPar  fnnrf'f'n  intor-nnlitnr'c 

(LFIs)  are  used  to  provide  the  aerodynamic  data 
for  flight  handlin'1  characteristics  of  the  trainer. 
Other  hardware  includes  two  disc  drives,  one  a 
backup  for  the  other,  a  Night  Only  Calligraphic 
Image  Generation  (NOCIG)  visual  sysi.em,  signal 
conversion  equipment  (SCE),  an  actual  F - 1 6  Delco 
Magic  362F  Fire  Control  Computer  (FCC),  a  mechano- 
recep^or  subsystem  (MRCS)  and  a  Sander's  display 
system  driving  three  CRTs,  a  light  per  and  three 
keyboards  at  the  F-lb  OFT  instructor  station. 

Student  Station.  The  cockpit  of  the  simulator 
is  an  exact  replica  of  an  F-16  aircraft  cockpit. 

It  includes  actual  aircraft  hardware  for  items 
such  as  the  side  stick  controller,  t.he  Head-Up  Dis¬ 
play  (HUD),  the  Radar/Electro-Optic  (RDR/EO;  sys¬ 
tem  an*.:  the  Fire  Control  Navigation  Panel  (FCNP). 

All  motion  is  simulated  with  a  merhanoreeptor 
cueing  system  (MRCS)  consisting  of  30°  reclined 
g-seat,  anti  - g  suit,  and  a  seat  shaker.  Unlike 
previous  simulators,  there  is  no  hydraulic  control 
loading  because,  just  as  in  the  aircraft,  the  sim¬ 
ulator  employs  a  fly-by-wire  flight  control  system 
which  interacts  via  signals  from  the  stick  and  rud¬ 
der  pedals'  force  transducers  throu  h  the  signal 
conversion  equipment  directly  to  the  flight  control 
system  software  model  to  provide  all  nec  ssary 
flight  dynamics.  Realistic  air:raft  and  environ¬ 
mental  sounds  are  reproduced  by  an  aural  cue  sys- 


1.  Power  Caoinet 

2.  Nerd  Compute  • 

3.  Hydraul ic 'Pneumatic  Pumps 

4.  tiVj  Display 

5.  Stud*.  i;l  Station  I  cock!  » t) 

6.  Cockpit  Peripheral  Cabinet 

7.  NVS  Support  Equipment 

8.  Instructor  Operating  Station  (IOS) 

9.  Mechanoreceptor  Cabinet  (MRC->) 

10.  76  M-BYTE  Dist.s 

F-16A  Trainer  Simulator 

On  several  ,  but  not  all,  trainers  a  Night  Vis¬ 
ual  System  (NVS)  provides  visual  cues  for  the  pi¬ 
lot  to  fly  take-offs,  approaches,  landings,  air-to 


ground  weanons  delivery,  air-to-a  r  intercepts,  and 
and  limited  air  refueling.  It  employs  a  conven¬ 
tional  hpamspl itter  CRT  display  with  an  instantan¬ 
eous  field  of  view  ( F J V )  of  ±22. 6C  horizontally  and 
+13°  and  -15°  vertically. 

Instructor  Operating  Station.  cinal  /,  nil  of 
this  is  orchestrated  at  the  instructor  rating 
Station  (IOS).  Designed  to  be  operated  by  one  in¬ 
structor,  the  IOS  includes  three  21  inch  -Ipha- 
numf-ric  CRT  displays  with  a  light  npi,  a  rada^/EO 
repeater  with  controls,  selected  flight  avionic: 

(  i  NYS  ronpptr-.  FunrHnnal  keyboards,  a 

three-axis  joystick,  and  a  communication  system. 
Each  CRT  can  provide  real-time  information  of  cock¬ 
pit  conditions,  system  malfunctions,  instrument/ 
navigation  procedures,  threats,  or  weapon  delivery 
results . 

J  LJi*.HT_  SOJ'JUAR  F 

The  software  is  organized  into  modules  which 
interact  in  a  real-time  simulation  ei.ironment 
(Fig  ,).  The  majority  of  the  software  is  ; 1 ttn 
in  FORTRAN  with  some  assembler  language  fc  setter 
efficiency. 

Each  module  is  small  (i.e.,  100  to  200  state¬ 
ments;  and  has  a  well  defined  function  such  as 
module  F500,  Drag  Coefficient.  The  Draa  Coeffi¬ 
cient  Module  calculates  the  total  drag  force 
coefficient  due  to  aerodynamics  including  the  basic 
aircraft  and  anv  perturbations  from  the  clean  con¬ 
figuration.  The  output  of  this  module  and  several 
other  aero  coefficient  modules  interracts  with  the 
atmosphere  module  and  equations  of  motion  module  to 
provide  the  flight  dynamics  for  the  simulator. 

Other  pv'di^es  are  similarly  grouped  to  provide  en¬ 
gine  dynamics,  navigation,  and  weapons  delivery 
simulation. 


Fig  3  Software  Organization 

' ■•.IMj 'LATpJl  .Pi VI.1  PJ P [PI 

Once  the  contract  is  let  for  the  simulator,  a 
large  effort  is  spent  in  gathering  data  (Fiq  4). 
Usually,  sufficient  data  does  not  exist  or  there 
are  anomalies  in  the  data.  Some  examples  for  the 
F-16  OFT  were:  insufficient  F-100  engine  data, 
incomplete  non  linear  rudder  and  flaperon  effects, 
and  incorrect  aeroelastic  modeling  equations.  Ad¬ 
ditional  data  must  then  be  generated  to  <over  void. 
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This  is  expensive  and  time  consuming.  Once  the 
data  hasp  is  correlated,  a  software  model  is  gen¬ 
erated,  debugged  and  refined. 

As  soon  as  possible  during  development,  highly 
qualified  piW(s)  should  be  brought  in  to  perform 
a  modified  factional  check  flight  of  the  simulator 
in  orJer  to  identify  a.iy  deficiencies  in  the  flight 
regime.  This  was  done  with  the  F-16  OFT  with  ex¬ 
cellent  results. 


Fig  4  Simulator  Development 

Once  Full  Scale  Development  (FSD)  and  Quality 
Control  (QC)  checks  are  complete,  extensive  inplant 
acceptance  tests  with  unit  pilots  begin.  Several 
halts  will  probably  occur  as  technical  problems 
arise  and  are  cleared  by  the  contractor.  If  all 
goes  well,  the  simulator  passes  several  critical 
tests  and  is  shippe..  lo  the  appropriate  training 
base.  If  all  does  not  go  well,  then  one  or  more 
technical  deficiencies  exist.  If  the  deficiency 
is  minor  and  does  not  effect  training  it  will 
probably  be  waived  for  shipment  an  I  fixed  prior  to 
the  ready  for  training  date  (RFT),  if  the  defi¬ 
ciency  is  major  then  it  must  Dl-  corrected  irolant. 
For  the  F-16  OFT  follow-on  simulators,  retesting 
consisted  of  condensed  testing  of  the  necessary 
areas  to  ensure  a  complete  and  accurate  system. 

irs/j’ROu  diirfs 

In  order  to  evaluate  the  complete  F-16  OFT  sys¬ 
tem  ove-'  4,000  pages  of  Acceptance  Test  Procedures 
(ATP)  were  written  and  i neerporotsr!  into  ten  vol¬ 
umes  .  One  of  these  volumes,  volume  seven,  con¬ 
tained  the  performance  evaluation  tests.  It  con¬ 
sisted  of:  weight  and  balance  checks,  atmosphere 
cnecks,  speed  power  tests,  engine  performance,  and 
both  static  and  dynamic  flight  characteristics. 

Weight  and  balance  tests  were  rather  straight¬ 
forward  readout'.,  of  gross  weights,  centers-of- 
gravity,  and  moments  of  inertia  for  several  F-16 
configurations.  Similarly,  the  atmospheric  checks 


evaluated  proper  temperature  lapse  rates,  air¬ 
speeds  and  mach  numbe*-  for  standard  (>15°C)  and 
nonstandard  days  (-20°C,  +40°C)  a.id  also  checked 
for  correct  wind,  turbulence,  and  gust  effect.. 

The  Speed  Power  tests  provided  the  first  de¬ 
tailed  comparison  of  the  performance  of  the  simu¬ 
lator  with  that  of  the  airc-aft.  Thpsp  tests  con¬ 
sisted  of  level  accel erations/decel erations ,  rates- 
of-clim.b,  and  times-to-cl  imb .  The  level  accelera¬ 
tion  tests  were  rur.  at  several  fixed  altitudes 
from  sea  level  to  50,000  feet  at  rn?vim.um  and  mili¬ 
tary  power  settings.  These  tosts  aro  highly  depen¬ 
dent  upon  an  accurate  engine  model  tor  thrust  and 
inlet  ram  drag,  and  an  equally  accurate  model  for 
aircraft  drag.  The  rate-of-cl imb  (P/C)  tests  were 
run  differently  in  the  OFT  than  in  the  aircraft. 
R/C  curves  were  aeierated  for  three  different 
gross  weights  at  saa  level,  10,000  feet,  and 
30,000  feet.  The  trainer  was  then  evaluated 
against  them  by  freezing  the  altitude  and  allowing 
the  |ji 1  o t  to  vary  the  true  airspeed  by  changing 
the  pitch  attitude.  Thrust  was  kept  constant  at 
ei ther n  il  itary  or  maximum  power  settings  and  gross 
weight  was  maintained  by  freezing  the  total  fuel. 
These  tests  became  mere  difficult  at  the  higher 
altitudes  and  gross  weights  due  to  its  sensitivity 
to  airspeed  and  pitch  changes  in  this  regime.  The 
time-to-cl imb  tests  were  more  dynamic  and,  there¬ 
fore,  similar  to  those  in  the  aircraft.  Two  tests 
were  run,  one  i n mi  1  i tary  power  and  one  in  maximum 
power.  Both  were  run  from  sea  level  but  timing 
was  not  started  until  the  trainer  passed  10,000 
feet  in  order  to  allow  the  pilot  to  stabilize  his 
climb  attitude.  The  pilot  initially  maintained  a 
constant  airspeed  (400  KIAS  for  mil  itary  ^ower;  550 
KIAS  for  maximum  power)  until  reaching  0.9  mach 
(mil  power)  or  1.5  mach  (max  power)  and  then  held 
constant  mach  until  completion  of  the  test.  Again, 
as  in  the  R/C  tests,  it  became  difficult  to  match 
the  data  at  the  high  altitudes  primarily  due  to 
lower  thrust  available,  pitch  sensitivity,  and 
pilot  technique.  Longitudinal  maneuvering  flight 
characteristics  were  evaluated  using  constant  air¬ 
speed  fixed  altitude  wind-up  turns  to  7  g's  to  de¬ 
termine  control  surface  variations  with  load  fac¬ 
tor.  Several  gross  weights  and  configuration  were 
analyzed  at  10,000  and  30,000  feet.  As  the  angle 
of  bank  and  the  g-loads  increased  most  pilots  had 
difficulty  maintaining  mach  number/airspeed  even 
with  altitude  frozen.  Repetition  improved  the 
pilot's  ability  and  the  test  results.  Longitudinal 
trim  checks  were  also  made  for  various  configura¬ 
tion  changes  such  as  gear,  flaps,  and  speed  brakes 
extension  and  retraction.  Lateral  trim  checks 
were  performed  by  steady-state  sideslips  in  which 
the  pilot  maintained  a  wings  level  sideslip  of 
constant  rudder  pedal  force.  The  rudder  pedal 
force  was  varied  up  to  full  rudder  deflection  for 
pach  of  several  airsppeds,  altitudes,  and  config¬ 
urations.  Results  for  sideslip  angle  (  ),  rudder 
deflection,  and  differential  flaperon  were  then 
compared  with  the  '’rcraft  data.  Lateral  trim  was 
also  checked  for  asymmetrical  loads. 

Dynamic  stability  flight  characteristics  were 
evaluated  by  first  comparing  the  longitudinal  short 
period  dynamics  with  those  of  the  aircraft.  A 
strip  chart  recorder  was  used  to  record  longitudi¬ 
nal  stick  force,  stabilator  position,  pitch  rate 
and  attitude,  angle  of  attack,  and  load  fac.->r 
while  the  automatic  test  features  of  the  computer 
were  used  to  input  a  pitch  force  doublet.  Tests 
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were  run  at.  three  basic  configurations,  three  dif¬ 
ferent  altitudes  and  several  airspeeds.  Similar 
procedures  were  used  to  evaluate  the  roll  mode. 
Again  the  autori.a tic  lest  feature  was  used  with  the 
strip  chart  recording  roll  force,  differential 
flaperon,  roll  rate,  and  roll  attitude.  From  these 
tests,  times  to  roll  90°,  180°  and  360°  for  several 
airspeeds,  weights,  and  configurations  were  com¬ 
pared  with  flight  test  data.  Finally,  the  Dutch 
Roll  mode  was  investigated  using  a  rudder  doublet 
(automatic  function)  and  recording  rudder  pedal 
force,  rudder  deflection,  yaw  rate,  roll  attitude, 
sideslip  angle,  lateral  acceleration,  roll  rate 

anu  u  t  f  fei  ciit'iul  MaptM  UH. 

Tests  were  also  performed  to  evaluate  1-g 
staMs  and  take-off  and  landing  performance.  The 
last  test  performed  was  an  overall  pilot  evalua¬ 
tion  of  the  entire  flight  envelope.  This  was  per¬ 
formed  by  a  highly  qualified  F-16  flight  test  pi¬ 
lot  whose  comments  were  used  to  further  improve 
the  simulator  data  base. 

7J  ST.  ,R  t.  SUITS/  PROBLEM  AH  LAS 

One  technique  which  has  provided  Air  Force  en¬ 
gineers  with  insight  into  potential  problems  in 
the  flight  handling  area  is  to  have  the  pilots 
"fly"  simple  profiles  during  their  first  time  in 
the  simulator.  This  allowed  discrepancies,  which 
might  not  otherwise  be  evident,  to  surface.  One 
such  discrepancy  was  noted  by  several  pilots. 

Their  comment  was  that  the  simulator  required  exces¬ 
sive  force  to  "unstick"  from  the  runway  during  ro¬ 
tation  and  lift-off.  Contributing  factors  were  a 
limited  field  of  view  visual  system  which  may  net 
have  provided  sufficient  cues  for  rotation  and 
lift-off  and  a  fixed-stick  controller  which  the 
pilots  were  not  accustomed  to  using.  The  primary 
cause,  however,  was  a  non-linear  load  curve  for 
the  main  landing  gear  which  was  approximated  by 
linear  equations.  Additional  break  points  were 
required  in  orde^  to  "smooth"  the  curve  to  fit  the 
actual  aircraft  performance. 

Another  discrepancy  was  noted  during  the 
landing  phase.  Just  prior  to  touchdown  almost  all 
pilots  entered  a  rolling  PIQ  (pilot  induce  oscil¬ 
lation)  on  their  first  landing  in  tbe  simulator. 

This  was  also  noted  initially  on  the  A-10  simula¬ 
tor.  As  the  number  of  approaches  and  landings  in¬ 
creased,  the  rolling  tendency  decreased  tn  zero. 

The  primary  cause  for  this  appears  to  be  a  lack  of 
visual  cues  near  the  terminal  phase  of  the  approach. 
The  pilot  then  overcontrols  the  roll  axis.  As  ex¬ 
perience  grows,  the  pilots  "learn"  to  slow  their 
roll  inputs  during  this  phase  to  effectively  eli¬ 
minate  this  problem. 

As  mentioned  previously,  it  was  difficult  to 
test  the  time-to-cl imb  above  40,000  feet  due  to  its 
sensitivity  to  airspeed  and  pitch.  Several  itera¬ 
tions  of  this  test  were  required  before  satisfac¬ 
tory  results  were  achieved.  In  this  rase*  the  data 
was  correct  but  the  test  procedures  were  difficult 
to  follow. 

Perhaps  the  area  of  highest  risk  was  in  the 
software  development  for  the  F-100  engine.  The 
actual  F-100  engine  in  the  F-16  aircraft  is  opera¬ 
tional  throughout  a  very  broad  flight  envelope  with 
rapid  changes  in  engine  demands.  In  order  to  prop¬ 
erly  nandle  this,  a  cycle  analysis  design  approach 
was  used  to  model  the  dynamics  of  each  engine 


section.  From  the  onset  this  effort  was  beset 
with  problems  ranging  from  missing  data,  especial¬ 
ly  transient  data,  to  personnel  changes.  Two 
years  were  required  to  perfect  trie  enyine  simula¬ 
tion. 

A  second  problem  area  of  trie  engine  develop¬ 
ment  existed  in  the  Back-up  Controller  (BUC)  which 
provides  the  fuel  flow  rate  to  the  engine  in  the 
event  of  failure  of  the  primary  Unified  Fuel  Con¬ 
trol  (UFC) .  Both  hardware  and  software  problems 
occurred  to  hamper  testing.  Once  the  hardware  was 
fixed,  pilot  comments  became  very  important  in 
isolating  software  problems  and  additional  data 
was  then  sought  to  correct  them. 

A_-J  0_A_  AJ  fi C R A;  J 

The  A-10  aircraft  is  a  single  place  close  air 
support  aircraft  built  by  Fairchild  Republic  Com¬ 
pany,  Farmingdale,  New  York.  It  is  powered  by 
General  Electric  TF34-GE-100  engines  having  a  max¬ 
imum  installed  thrust  of  9,000  pounds  per  engine. 
The  maximum  gross  weight  of  the  aircraft  is  47,500 
pounds.  The  engine  is  a  high  bypass  turbofan  with 
a  single  fan  rotor,  fourteen  stage  compressor  and 
six  turbine  stages  (See  Fig  5).  The  aircraft  has 
a  length  of  53.5  feet,  a  wing  span  of  57.5  feet, 
and  a  wing  area  of  405  square  feet.  The  wing  air¬ 
foil  is  an  NACA  6716  inboard  of  the  landing  gear 
pods  and  an  NACA  6713  outboard.  The  wing  contains 
four  panel,  three  position  flaps,  aileron  and 
aileron  tab  surfaces,  and  eight  pylon  weapon  sta¬ 
tions;  three  additional  pylon  stations  are  located 
on  the  fuselage.  The  ailerons  consist  of  upper 
and  lower  panels  which  also  function  as  speed- 
brakes  when  moved  symmetrically.  The  empennage 
consist*,  of  twin  vertical  rudders,  a  horizontal 
stabilizer,  and  elevators  with  elevator  tab  sur¬ 
faces.  The  horizontal  stabilizer  and  vertical 
stabilizer  are  NACA  64A013  airfoils.  The  armament 
system  includes  a  4,000  round  per  minute  30mm 
seven  barrel  gun.  The  flight  control  system  is 
designed  to  operate  with  single  or  dual  hydraulics 
shut  down;  the  latter  case  is  “ailed  Manual  Rever¬ 
sion.  Without  hydraulic  power,  roll  control  me¬ 
chanically  transfers  from  aileron  surface  control 
to  aileron  tab  control  by  means  of  a  roll  tab 
shifter  device  near  the  control  surfaces.  Mechan¬ 
ical  disconnect  devices,  in  both  the  pitch  and 
roll  control  axes,  free  the  control  stick  to  oper¬ 
ate  in  one  of  two  separate  paths  in  both  pitch  and 
roll  in  the  event  of  a  mechanical  linkage  jam. 
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A-JOA_ TRAINER  SIMULATOR 

Reflectone,  Inc.,  of  Tampa,  Florida  builds  the 
A-10A  Operational  Flight  Trainer  for  the  Air  Force 
through  an  initial  1976  contract  with  the  Simula¬ 
tor  Systems  Program  Office.  The  principal  mission 
of  this  trainer  is  to  provide  the  capability  of 
procedures  and  proficiency  training  to  pilots  re¬ 
quired  to  fly  the  A-10A  aircraft  in  fulfillment  of 
its  mission  to  navigate,  seek  out  and  destroy 
ground  targets.  The  trainer  provides  the  means  of 
developing  proficiency  in  cu1  phases  of  instrument 
flight,  including  ground  operations,  takeoff,  en- 
route  navigation,  holding,  penetration,  approach, 
and  landing  under  hotii  normal  and  emergency  con¬ 
ditions.  The  visual  system  permits  practice  in 
normal  and  emergency  procedures  under  simulated 
night  visual  conditions.  Experience  will  also  be 
gained  in  selection  and  release  procedures  asso¬ 
ciated  with  the  basic  armament  system  and  simulat¬ 
ed  electronic  warfare  (EW)  equipment. 

TRAINER  HARDWARE 

The  trainer  hardware  and  floor  layout  are 
shown  in  Figure  6.  The  primary  computer  capabil¬ 
ities  consist  of:  Three  System  Electronics  Labo¬ 
ratory  (SEL)  32/55  computers  (Units  1),  an  MDEC 
Vital  IV  system  consisting  of  a  Varian  Image  Gen¬ 
erator  Processor  and  Visual  Display  Unit  (Units  7 
and  2),  an  LSI-11  Minicomputer,  aft  of  the  cockpit 
station,  for  control  loading  modeling  (Unit  8). 
Additional  hardware  systems  are  the  Instructor 
Station  (Unit  3),  the  Electronic  Warfare  Simula¬ 
tion  Cabinet  and  Console  (Unit  4),  Audio  Cabinet 
System  (Unit  5),  and  Hydraulic  and  Electrical 
Power  Equipment  (Unit  6). 


systems  of  an  aircraft.  Tr.e  traditional  m°thod 
has  been  a  system  consisting  of  sDosnr<; ;  mechanics , 
hydraulics,  drive  electronics,  and  modeling  ties 
together  with  some  I/O  interface  to  a  host  CPU. 

In  this  design,  the  modeling  electronics  (basical¬ 
ly  an  analog  computer)  was  replaced  with  a  digital 
computer.  The  control  system  modeling  of  the  air¬ 
craft  is  then  performed  by  the  digital  computer 
software.  I/O  interchange  to  the  host  SEL  comput¬ 
er  is  now  performed  by  direct  memory  address  (DMA) 
exchanges  between  the  LSI-11  digital  control 
loader  compute*'-  anr^  t^°  hnct  ppii 
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Fig  1  Digital  Control  Loading  Subsystem 

The  propulsion  module  consists  of  approxi¬ 
mately  600  lines  or  Fortran  code  and  is  executed 
at  10Hz  with  rectangular  integration.  Its  primary 
outputs  are  net  thrust  and  ram  drag  to  the  aero¬ 
dynamics  module,  fuel  flow  to  the  fuel  systems 
module,  and  thrust  and  engine  speed  to  an  aural 
cue  module.  The  flight  module  consists  of  approx¬ 
imately  550  1  i  ics  of  code  and  performs  calcula¬ 
tions  and  summation  of  all  aerodynamic  forces  and 
moments  acting  at  the  aircraft  centcr-of-gruv ity . 
These  forces  and  moments  are  a  function  of  config¬ 
uration,  control  surfaces,  engine  flrust,  angle  of 
attack,  sideslip,  body  angular  rates,  and  environ¬ 
mental  conditions.  The  module  data  is  in  approxi¬ 
mately  60  percent  tabular  and  40  portent  equation 
form.  T fie  flight  equations  or  motion  arc  computed 
at  20Hz  using  a  rectangular  integration  sJunc, 


Fig  6  A-10A  Simulator  Facility 

1  t  Mi!  I  W;-Rj 

The  software  is  primarily  coded  in  Fortran 
with  some  assembly  coding  for  the  non-real-time 
programs.  The  trainer  S/W  system  contains  approx¬ 
imately  300,000  lines  of  code  of  which  all  are 
disc  stored.  The  flight  control  system  for  the 
A- 1 0  trainer  simulator  is  a  digital  control  ’oad- 
ing  system  (DCL)  designed  and  built  inhouse  by 
Reflectone,  Inc.,  of  Tampa,  Tlorida.  The  PCL  rep¬ 
resents  a  novel  method  for  'inflating  t fie  control 


The  total  system',  design  data  base  i«  best 
appreciated  by  realizing  that  a  top  level  document 
listing  the  design  references  contains  over  500 
individual  entries.  T fie  fliqht  systems  list  ’’s 
30-40  percent  of  tin  total  and  includes  flight 
test  reports  from  Fairchild  Republic  Aircraft  anj 
the  Air  Force  Flight  Test  Center  as  well  as  air¬ 
craft  technical  orders,  engineering  reports  and 
drawings.  Additional  government  report-,  from  the 
Air  Force's  Tactical  Air  Command,  Flight  Dynamics 
laboratory,  and  Human  Resource.  Laboratory  were 
also  included  in  the  <it'>.'igr.  data  base. 
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The  aerodynamic  and  propulsion  data  base  is 
currently  a  mix  of  wind  tunnel,  flight  test  and 
data  variants  due  to  pilot  evaluation.  The  basic 
set  of  da  i.n  and  equations  are  based  upon  a  cruise 
configured  A-10A.  Data  changes  due  to  configura¬ 
tion  changes,  external  stores,  power  ancj  hydraulic 
status  are  generally  modeled  as  incremental  effects 
to  the  cruise  configured  aircraft.  Aerodynamic 
effects  due  to  buffet,  ground  effects,  and  asym¬ 
metric  controls  and  mal functions  are  also  modeled. 
Wing  stall  conditions  are  computed  on  each  wing 
side  individually  based  on  AOA  and  tip  roll  rate, 
bide  force  due  to  rudder  cind  lift  force  due  to  ele¬ 
vator  are  modeled  as  geometric  multipliers  of  yaw¬ 
ing  moment  due  to  rudder  and  pitching  moment  due 
to  elevator  respectively.  Reductions  in  lift  and 
a  nose  down  pitching  moment  due  to  the  absolute 
value  of  sideslip  are  also  included.  These  values 
are  15  and  10  pounds  of  lift  coefficient  and  pitch¬ 
ing  moment  respectively  per  degree  of  sideslip. 

The  original  aerodynamic  data  S/W  load  was 
based  almost  exclusively  upon  wind  tunnel  data 
primarily  obtained  from  1/10  scale  model  tests  of 
the  A- 10  in  the  NASA  Langely  7x10  ft  HST  and  the 
Ames  12  ft  PWT.  This  data  resulted  in  poor  lift/ 
moment  effects  due  to  power  induced  flow  particu¬ 
larly  with  secondary  control  surfaces,  i.e.,  speed- 
brake  and  flap  deflection.  Wind  tunnel  roll  con¬ 
trol  effectiveness  for  the  cruise  configuration 
was  also  excessive  but  identified  and  documented 
during  flight  test.  Much  of  the  flight  test  deriv¬ 
atives  identified  in  AFFTC-TR-77-1  were  incorpo¬ 
rated  into  the  simulator;  however,  these  deriva¬ 
tives  primarily  addressed  the  low  angle  of  attack 
regime  in  a  cruise  configuration.  The  side  force 
due  to  aileron  and  normal  force  due  to  elevator 
were  incorrect  and  not  incorporated;  the  lateral 
accelerometer  location  had  been  mislocated  in  the 
computer  derivative  extraction  program  resulting  in 
the  incorrect  side  force  derivative;  wing  to  ele¬ 
vator  center-of-pressure  geometric  constraints  for 
C[_v  computation  had  also  not  been  entered  i ’to  the 
program.  The  point  in  mentioning  these  data  dis¬ 
crepancies  is  that  the  A-10  aircraft  program  had  a 
huge  aerodynamic  data  base  with  inaccuracies  and 
conflicting  data  which  required  considerable  effort 
to  determine  and  delete  in  order  to  arrive  at  a 
data  base  acceptable  for  trainer  simulation  devel¬ 
opment,  Although  a  huge  data  base  existed,  there 
were  also  considerable  gaps  that  needed  definition; 
some  of  these  were:  (1)  external  stores  mass  and 
inertial  properties,  and  rack  and  pylon  locations, 
(2)  definition  of  control  surface  hinge  moment 
characteristics  relevant  to  cockpit  control  forces 
during  Manual  Reversion  Flight  (no  hydraulics). 

TEST  ME  TODS. 

This  will  be  described  in  a  broad  sense  relat¬ 
ing  to  test  effectiveness  and  efficiency.  Air 
Force  simulator  acceptance  testing  has  tradition¬ 
ally  been  very  time  constrained  by  comparison  with 
aircraft  testing.  In  this  regard,  test  management 
coupled  with  test  planning  and  sequencing  along 
with  test  techniques  and  data  were  very  critical 
to  test  accomplishments.  In  reflection,  insuffi¬ 
cient  government  involvement  during  contractor  In- 
plant  testing  of  the  first  article  and  on  abbrevi¬ 
ated  Test  Readiness  Evaluation  prior  to  large  seal* 
start  of  government  testing  resulted  In  consider¬ 
able  misunderstanding  between  Air  Force  and  the 
contractor  during  subsequent  A-10  simulator  tost-: 
ing.  Also,  insufficient  use  was  made  of  TAC/Tost 


Pilots  during  contractor  Inplant  testing;  results 
during  this  period  were  also  not  sufficiently  doc¬ 
umented  or  remedied.  In  addition,  simulation  pe¬ 
culiar  test  procedures  for  determining  system  per¬ 
formance  were  not  sufficiently  detailed  to  allow 
efficient  and  large  scale  official  government  test¬ 
ing.  Detailed  flight  performance  was  attempted 
prior  to  a  thorough  computer  systems  operation 
;  “lockout  resulting  in  poor  flight  system  test  ef¬ 
ficiency.  Many  computer  halts  and  dropouts  were 
encountered  which  hampered  the  fight  performance 
evaluation . 

TEST  EQUIPMENT 

Calibrated  force  gages  (load  cells)  were  used 
by  the  government  to  calibrate  S/W  values  for  cock¬ 
pit  control  forces.  A  single  eight  pen  stripchart 
recorder  (SCR)  was  used  to  provide  a  continuous 
record  of  flight  performance  and  dynamics;  a  sec¬ 
ondary  and  somewhat  unplanned  evaluation  fallout 
was  its  use  in  identifying  sources  of  computer 
halts  and  interrupts.  An  X-Y-Y  plotter  was  also 
used  and  was  extremely  useful  in  generating  flight 
control  gearing  curves  of  force  and  position.  The 
plotter  graphically  portrayed  undesirable  control 
force  to  position  "ratcheting"  associated  with  the 
digital  control  system  implementation.  This  ana¬ 
log  test  equipment  was  pmvided  drive  signals  by  a 
S/W  routine  containing  120  internal  computer  sig¬ 
nals;  they  were  converted  for  display  through  DACs 
updated  at  2Hz.  Additional  test  display  aids  were 
the  primary  cockpit  instruments  repeated  on  the 
instructor  console,  i.e.,  ADI,  altimeter,  VVI,  air¬ 
speed,  and  HSI;  these  instruments  were  also  re¬ 
peated  graphically  on  Sanders  CRT  display  pages 
along  with  engine  parameter  bar  graphs.  This  in¬ 
formation  was  very  helpful  in  communicating  and 
understanding  pilot  evaluations  and  concerns. 

There  were  additional  CRT  display  pages  that  were 
very  helpful  for  the  evaluation;  a  control  param¬ 
eter  page  showing  cockpi t- control  and  surface  po¬ 
sitions  and  a  flight  parameters  page  listing 
weight,  configuration,  airspeed,  engine,  environ¬ 
ment,  and  acceleration/rate/attitude  response. 

TEST  TECHNIQUES 

Procedures  and  test  results  for  simulation 
qualification  were  contained  in  a  document  called 
the  Acceptance  Test  Document.  For  flight  systems 
testing  this  document  contained  a  large  assortment 
of  flight  tost  results  primarily  obtained  from  two 
preproduction  A-10A  aircraft  S/N  73-01665  and  73- 
01667.  (These  flight  test  report  results  are  ref¬ 
erences  1  and  2.)  T he  test  techniques  used  in 
simulation  testing  paralleled  flight  test  tech¬ 
niques  to  a  large  degree.  Flight  and  simulator 
sidetllp  were  done  with  steady  heading  rather  than 
•wings  level.  For  longitudinal  and  lateral-direc¬ 
tional  dynamics,  actual  flight  control  surface 
signatures  were  duplicated  and  used  to  drive  sim¬ 
ulator  response.  Freezing  altitude  during  many  of 
these  tests  reduced  pilot  workload  and  improved 
data  correlation;  this  could  be  done  without  af¬ 
fecting  the  calculation  or  display  of  velocity  or 
attitude.  Prior  to  these  system  performance  tests, 
lengthy  tests  had  to  be  performed  to  validate  the 
environment,  1,e,,  the  atmospheric  model,  pressure 
sensitive  Instruments,  and  weight  and  balance. 

One-G  stalls  were  determined  from  SCR  data  of  lift 
coefficient,  G's  and  AOA,  These  AOA  values  corre^ 
lated  well  with  a  more  easily  observable  VVI  break 
on  repeater  Instruments  during  one-G  stalls. 


elude  this  yaw  SAS  change. 


Maneuvering  stability  and  stall  pitch  out  were 
tested  during  windup  turn';  at  constant  airspeed. 
Post-stall  gyration,  spin,  and  spin  recovery  were 
evaluated  against  specific  flight  time  history 
signatures.  An  attempt  to  replicate  stall/post- 
stail  flight  control  inputs  were  done  manually  with 
a  control  input/stick  box  diagram  and  written  ma¬ 
neuver  descriptions . 

TESJ_ PROBLEMS 

A  number  of  nprfnrmanrp  differences  were  ob¬ 
served  during  testing  of  the  No.  1  simulator  unit. 
As  a  consequence  of  this  testing,  approximately 
1,500  test  discrepancies  were  identified,  of  which 
300  were  against  flight  systems;  of  these,  40  per¬ 
cent  were  system  logic  errors,  the  remainder  were 
system  performance  errors  ranging  from  the  manner 
in  which  rudder  serve  valves  shutdown  during  a 
single  channel  SAS  malfunction  to  the  dynamic  re¬ 
sponse  of  the  VVI . 


Attitude  Changes.  Slow  yaw,  pitch,  end  roll 
control  attitude  changes  were  observed  on  instru¬ 
ments  and  the  visual  system  following  a  controls 
free,  careful  one-G  trim  setup.  The  motion  drift 
was  caused  by  fluctuating  surface  control  positions 
from  the  LSI-11  digital  control  loader  into  FFLilL, 
the  aerodynamics  module,  causing  subsequent  atti¬ 


tude  changes.  It  was  primarily  solved  by  creating 
a  S/W  statistical  automatic  fine  tuning  •cap  for 
toe  flight  control  system  in  pitch,  roll,  and  yaw 
to  account  for  small  varying  null  positions  due  to 
the  hardware.  This  logic  looks  at  stick  and  pedal 
velocity,  and  force  over  64  sampling  periods  (less 
than  one  second  at  j  sampling  rate  of  156Hz)  then 
statistically  averages  the  values  and  uses  them  as 
new  and  current  bias  and  null  forces  for  se^vo- 


Take-Off  Rotation  Speeds.  Two  problem* 
existed  which,  when  recognized  as  being  related, 
were  both  corrected  with  a  single  design  change. 
The  first  was  the  minimum  rotation  speed  at  take¬ 
off  which  was  higher  than  the  aircraft.  The  other 
problem  was  an  excessive  pitch  down  with  a  throttle 
chop  (the  aircraft  has  negligible  initial  pitch 
change  with  throttle  motion).  A  review  of  the 
software  model  engine  1 ine-of-thrust  and  design 
data  showed  that  the  pitching  moment  associated 
with  tailpipe  curvature  ('-i.'i  degrees  up)  hod  nut 
been  accounted  for.  Inclusion  of  this  effect  in 
the  model  solved  both  the  higher  rotation  speed 
and  the  initial  pitch  up  with  a  throttle  chop. 

An  adued  benefit  was  the  readjustment  (f  Cmo  values 
bauk  to  magnitudes  published  in  flight  test  re¬ 
ports.  Prior  to  recognition  of  the  tailpipe  cur¬ 
vature  effect,  the  flight  values  of  Cmo  had  been 
considerably  modified  to  correct  trim  elevator 
differences  between  the  simulator  and  flight 
resul ts. 

Pitch/Roll  Sensitivity.  This  problem  still 
exists  to  some  degree  on  the  simulator;  it  is  an 
excessiv2  attitude  response  for  small  pitch  and 
roll  control  inputs.  Previous  changes  to  the 
simulator  have  not  corrected  this  issue;  the 
aileron  actuator  model  had  been  slowed  down  from 
•i  50  millisecond  first  order  lag  to  a  120  milli¬ 
second  lag;  in  addition,  aileron  control  surface 
aerodynamic  effectiveness  was  reduced  for  deflec¬ 
tions  under  five  degrees.  Design  data  (wind  tunnel 
and  flight  test)  were  generally  lacking-only  show¬ 
ing  roll  effectiveness  for  much  larger  control 
deflections. 


value  commands.  A  secondary  fix  involved  a  sepa-  Flight  Test-Simulator  Recommended  improvements 
rate  hold  (clamp)  circuit  for  the  control  surfaces 

to  eliminate  small  signal  disturbances.  Stick  In  order  to  improve  simulator  quality  and 

position  and  stick  rate,  surface  control  and  trim  test  effectiveness,  we  recommend  the  following 

positions  are  used  in  the  hold  circuit  logic.  improvements  in  the  data  buse:  (1)  SurTace  au- 

When  these  positions  and  velocity  signals  are  with-  thorities  and  maximum  rates  relative  to  airloads/ 

in  a  small  magnitude  level,  a  new  current  trim  airspeed,  (2)  Identification  of  any  control  sur- 

position  is  used.  face  effectiveness  reduction  for  small  deflections 

(e.g.,  3  degrees),  (3)  Cockpit  control  trim  motor 
Tracki ng/Strafing .  The  pilot  control  workload  transient  characteristics  relative  to  surface  and 
associated  with  capturing  and  maintaining  t fie  Head-  stick  motion  (e.g.,  no.  of  clicks  to  r j  1 1  author- 

Up-Display  (HUD)  gun  cross  on  the  target  during  ity),  (4)  Flight  control  system  damning  and  fro- 

acquisition  and  tracking  was  excessive  by  compari-  quency  characteristics  for  normal  anu  deqraded 
;on  with  the  aircraft  workload  required.  This  was  hydraulic  and  electrical  power  operation,  (5)  Post 

an  early  recognized  deficiency  of  the  simulator  stall  dynamics  and  departure  characteristics.  (6) 


which  was  also  one  uf  the  last  problems  corrected; 
it  was  solved  by  deviating  from  design  yaw  stabil¬ 
ity  augmentation  criteria,  i' he  design  criteria 
called  for  an  aileron  to  rudder  interconnect  (ARI) 
lain  of  .2  dc  jrees  of  rudder  per  degree  above  2uU 
knots  along  with  a  >aw  rate  washout  circuit  of 
four  seconds  (4s/4s*l).  This  SEl  computer  circuit 
was  modified  during  pilot  evaluation  to  a  fixed 
value  at  all  airspeeds  of  .8  degrees  per  DPS  for 
the  ARI  rate  damper.  This  change  not  cr.ly  result¬ 
ed  in  clearing  this  discrepancy  but  also  another 
characterized  by  e*cessive  yaw  and  roll  during 
final  landing  approach.  This  change  appears  to  be 
a  situation  where  .owe  undetected  (negative)  char¬ 
acteristic  of  the  simulator  was  corrected  by  an 
intentional  deviation  (negative)  resulting  in  a 
(positive)  acceptable  performance  result.  Prior 
to  snaking  this  change,  the  visual  brightness  of  the 
gun  strafing  target  was  poor  and  was  subsequently 
brightened.  This  visual  change  reduced  pilot  work¬ 
load  but  was  not  o?  a  sufficient  nature  to  pre¬ 


Control  response  sensitivity  for  the  power  approac n 
configuration,  (7)  FI ight  test  document  expans k>n 
of  written  descrintions  of  aircraft  peculiar  char¬ 
acteristics  even  if  the  data  base  doe  not  fully 
e*plain  the  phenomena,  (8)  Identification  of  con¬ 
flicting  data  between  viable  sources  (flight  test 
vs.  contractor),  and  (9)  improved  communication 
between  the  aircraft  test  immunity  and  the  train¬ 
er  simulation  community. 

I  •  :  Ml  jy  '-J 

The  following  recoew  endat ions  are  suggested 
in  simulator  and  flight  test  methods  to  improve 
simulator  efficiency  and  quality:  (1)  Combine 
ADA  and  control  surface  trim  definition  with  speed 
power  test  during  aircraft  flight  and  simulator 
testing.  (2)  Freeze  simulation  altitude  while 
allowing  accelerations,  velocities,  and  attitudes 
to  change-to  reduce  pilot  workload  during  speed 
stability,  maneuvering  stability  and  flight  dynare- 
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ics  tests;  (?)  Better  describe  how  flight  test 
configuration  trim  changes  are  performed  (e.g., 
constant  attitude,  power,  altitude).  There  is  an 
inherent  potential  for  simulation  dynamic  cue  mis¬ 
match  (e.g.,  visual,  cockpit  instruments,  flight 
modules  operating  at  different  computational  update 
rates)  and  fewer  (than  aircraft)  cues  available  in 
a  simulator.  Because  of  this,  additional  cockpit 
instrumentation  flight  characteristics  are  re¬ 
quired.  This  cue  data  should  be  time  tagged  to 
normal  aircraft  response  parameters  (e.q.,  a  video 
camera  and  recorder).  We  recommend  a  cyclic  Bode 
frequency  responre  test  at  fixed  airspeeds.  Flight 
test  pilots  should  also  "fly"  the  simulator  as 
soon  as  possible  to  identify  deficient  areas. 

Rather  than  perform  initial  flight  simulation 
testing  in  a  segmented  classical  sense,  we  suggest 
that  a  few  aircraft  tests  be  performed  speci fical ly 
structured  for  simulation  use.  These  tests  would 
be  combined  expanded  functional  check  flight  com¬ 
plied  with  operational  scenarios  with  a  fully  in¬ 
strumented  aircraft  containing  pilot  voice  record¬ 
ing  and  a  video  recorder  for  instrument  response 
information.  This  same  test  profile  can  then  be 
flown  on  the  simulator  in  the  same  manner  again 
recording  pilot  comments  and  measuring  simulator 
performance.  A  method  of  this  nature  would  allow 
early  identification  of  significant  simulation 
flight  system  deficiencies. 

PLANNING  METHODS 

Air  Force  acceptance  of  First  Article  Simula¬ 
tor  Systems  has  been  lengthy  with  unplanned  exten¬ 
sions  of  test  schedules  because  of  the  unantici¬ 
pated  large  number  of  serious  Test  Discrepancies. 
This  problem  can  be  minimized  if  certain  concepts 
are  emphasized  such  as:  (1)  Providing  briefings 
of  system  operation  and  flight  characteristics  to 
contractor  personnel,  (2)  Showing,  and  possibly 
demonstrating,  the  aircraft  to  contractor  person¬ 
nel,  (3)  Identifying  data  base  milestones  within 
the  contract,  better  organize  the  design  documents, 
especially  their  accuracy,  currency,  and  '•omplete- 
ness,  (4)  Assuming  that  additional  aircraft  flight 
data  would  be  necessary  for  design,  thereby  dedi¬ 
cating  some  flight  missions  for  simulation  data 
purposes,  (5)  Developing  <  "prototype"  flight 
station  as  a  design  tool -between  the  user  pilots 
and  design  personnel  -  for  early  correction  of  sys¬ 
tem  operation  and  flight  performance  problems. 

i RANsrrR  or  i raining 

The  Air  force  spends  sizable  sums  of  money  to 
replica  aircraft  systems  opention  and  displays, 
flight  performance  and  dynamics,  and  cc  ‘responding 
notion  anu  visual  c  s .  Questions  arise  concern¬ 
ing  the  need  for  th  large  design  and  cost  effort, 
the  authors  telieve  that  high  fidelity  flight  per¬ 
formance  and  dynamics  an*  necessary  but  not  suffi¬ 
cient  for  high  Ira  ns  for  uf  Training.  Some  addi¬ 
tional  and  necessary  factors  are:  (1)  Training 
scenarios  that  reflect  an  operational  mission, 

(?)  Induced  psychological  stress,  and  (?)  Suffi¬ 
cient  visual  /motion  cu  *s  appropriate  for  a  ta.uk. 

If  system,  training  requirements  are  not  cle  rly 
defined  by  the  USER,  or  misunderstood  hy  the  pro¬ 
curing  agency,  a  trainer  can  easily  be  developed 
with  capabilities  not  totally  suited  for  training. 
Both  the  A-1G  and  F - 1 6  simulators  have  high  fidel¬ 
ity  flight  performance  and  weapons  capabilities; 


however,  a  realistic  ground  attack  or  visual  land¬ 
ing  approaches  can  hardly  be  trained  p  uperly 
because  of  their  single  window  visual  systems. 
Without  the  oossibility  of  an  accident  occurring, 
it's  difficult  to  obtain  appr  priate  stress  and 
workload  levels.  Minor  system  performance  errors 
can  have  large  impacts  to  positive  transfer  of 
trailing  for  new  students  who  tend  more  t°  gener¬ 
alize  s.ngle  issues  into  poor  total  system  perfor¬ 
mance.  Transfer  of  training  for  the  Manual  Rever¬ 
sion  flight  now  for  the  A- 10  is  expected  to  be  less 
than  desirable  at  this  Lime  bec^u^e  fuiCc/feel 
characteristics  for  trim,  configuration,  and  power 
changes  do  not  sufficiently  reflect  'he  aircraft. 
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ABSTRACT 


-This  paper  discusses  the  Military  Airlift  Command's  (MAC's)  current  and  future  aircrew 
training  programs.  Emphasis  is  placed  on  the  aircrew  training  program  structure,  the  use 
of  simulation,  and  the  benefits  derived  from  simulations.  The  paper's  purpose  is  to  pro¬ 
vide  a  brief  history  of  MAC  simulation  and  discuss  the  command's  training  philosophy. 
Current  programs  and  their  problems  are  outlined  with  a  brief  look  at  MAC'S  future  plans, x 


Aircrew  Training  devices  in  the  Military  Air¬ 
lift  Command  (MAC)  are  an  integral  par.  of  the 
training  programs.  Without  simulation  the  over¬ 
all  readiness  of  MAC'S  forces  would  be  hindered 
in  today's  environment.  Increased  fuel  costs  and 
decreased  flying  hours  make  additional  aircraft 
training  difficult.  Simulation  must  be  used  to 
Fill  the  void. 

Thp  mission  of  the  Military  Airlift  Command 
is  to  maiuain  the  military  airlift  system  and 
services  in  a  constant  state  of  readiness.  It 
must  be  able  to  respond  immediately  to  any  loca¬ 
tion  on  the  globe.  In  order  to  maintain  this 
state  of  readiness,  an  effective  aircrew  training 
program  is  essential.  Since  the  ear-y  '50s,  MAC 
has  relied  heavily  on  flight  simulation  in  train¬ 
ing  its  aircrews.  As  ti.e  simulation  industry 
evolved,  training  programs  took  advantage  of  the 
advances.  Today  simulation  is  essential  to  MmC's 
readiness.  Training  programs  are  structured 
using  the  instructional  systems  development  (ISDl 
approach  to  produce  efficient  and  effective  pro¬ 
grams.  This  produces  a  mix  of  classroom,  learn¬ 
ing  center,  simulator,  and  flying  activities. 
Beyond  training  critical  emergency  procedures 
whicn  cannot  be  trained  in  the  aircraft,  crews 
receive  training  in  crew  coordination,  normal 
procedures,  noncritical  emergency  procedures,  ar  1 
instrument  flight  through  simulation.  The  time 
logged  in  tie  simulator  is  credited  toward  ini¬ 
tial"  qualification  and  currency  in  MAC  aircraft. 
Simulation  allows  MAC  to  effectively  perform  its 
day-to-day  and  wartime  mission  with  increased 
efficiency  in  crew  performance  and  savings  in 
flyinn  hours. 

MAC'S  airland  mission  is  similar  that  of 
the  commercial  airlines,  therefore,  their  train¬ 
ing  programs  are  also  similar.  In  addition,  *1AC 
has  an  additional  air  refuel  i  g  and  a  tactical 
low  level /ai rd’fp  mission  which  nave  unique 
training  requirements.  Although  the  mission  is 
similar,  the  experience  level  cV  the  jvraQe  MAC 
S tudeo i  is  much  lower  than  that  of  the  average 
airline  student.  The  average  MAC  cupil  it  student 
his  just  "unshed  undergraduate  pilot  training 
and  ^as  approvirmtely  150  tu  200  hours.  ’he 
normal  air 'raft  commander  Candidate  usually  has 
100d  ’.tfOO  naurs  which  is  much  lower  true  the 
dverj.p  jirli'-e  captain  candidate. 


HISTORY 

In  the  early  50 ' s ,  the  Military  Air  Transport 
Service  (MATS),  the  forerunner  of  MAC,  used 
flight  simulators  to  train  transport  aircrews 
MATS'  first  simulator,  a  C-97  aircraft  flight 
simulator,  was  delivered  on  15  October  1952  and 
became  operational  on  1  December  1952.  By  June 
of  1953,  MATS  had  one  C- 124  and  three  C-97  flight 
simulators  operational  at  its  West  Palm  Beach 
International  Airport  facility..  Training  hours 
in  these  simulators  were  substituted  for  a  por¬ 
tion  of  the  total  flying  hours  required  to  train 
students  in  these  aircraft. (1)  In  addition  to 
initial  and  upgrade  training,  the  Continental 
Division  ( responsiole  for  this  training)  was 
tasked  to  establish  simulator  refresher  training 
courses  in  the  C-97  and  1-124  for  the  entire  com¬ 
mand.  These  courses  were  to  be  used  to  provide 
training  for  operational  aircrews.  The  training 
wa s  designed  to  standardize  procedures  and  teach 
additional  emergency  procedures  which  could  not 
be  accomplished  during  actual  flight 
operations.  (2) 

The  first  aircrew  training  courses  were  more 
or  less  experimental  in  regard  U  the  mix  of  air¬ 
craft  and  simulator  time.  The  simulator  proved 
to  be  a  valuable  supplement  to  the  aircraft  for 
aircrew  training.  But  the  questions  of  fre¬ 
quency,  purpose,  and  the  manner  in  which  the 
simulator  should  Le  u_ed  had  not  been  fully 
explored. 

IV a  ij l  i_i  iq  _E_f  fecti  v ene s s_  study 

lo  partially  answer  some  of  the  questions 
concerning  the  use  of  simulators,  the  *i*s" 
tra'iiinc  effectiveness  study  for  MATS  was  u;s. 
ducted  Py  ihe  Crew  Research  Labcnatcry  ef  the  Air 
Research  and  Development  Command.  This  study  was 
undertaken  to  compare  the  effectiveness  nr  vari¬ 
ous  typ  xs  of  transition  train;  \g  for  the  C -if. 
i n? ft*  different  mixes  of  amnn?  and  simulator 
til*-  were  evaluated  along  with  three  different 

sequences  of  simulator  -ractu  . . 

craft,  ni*e<l,  and  after  a  l  rc  rat  t. .  (  3)  The  major 
finding'  were:  1)  the  lew  an^ra't  high  Simu¬ 

la  to*  time  c  ■■edition  was  clearly  ;  rt  t-n  .»•  to 
dthe-'  Cund  1  f  1  or;S  tor  student  aircraft  t  tmnut  0«  i  - 
i‘ )  ’he  si  SMiI.i*.  or  first  sequence-  w.«  cha'ly 
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superior  to  other  sequences  3)  the  study  further 
concluded  that  a  relatively  equal  nix  of  simu¬ 
lator  and  aircraft  time  not  only  proved  to  be  the 
most  cost  effective,  but  yielded  the  same  profi¬ 
ciency  results  as  training  with  more  aircraft  and 
less  simulator  time. (4}  A  similar  study  was  con¬ 
ducted  for  the  C - 124  aircraft  and  yielded  like 
results. (6) 

Since  these  Qarly  days  of  crew  training  at 
West  Palm  Beach,  simulators  have  played  a  key 
role  in  MAC'S  overall  training  program.  As  a 
command,  MAC  has  always  operated  with  simulators 
as  a  means  of  reducing  actual  aircraft  flying 
training  time.  However,  it  is  difficult  to  doc¬ 
ument  any  savings  because  the  command  never  oper¬ 
ated  without  simulators. 

The  majority  of  MAC'S  existing  training  pro¬ 
grams  are  based  on  those  programs  developed  in 
the  early  70 1 s  and  utilize  simulators  procured  in 
the  early  60 ' s .  These  programs  and  devices  met 
MAC'S  needs  in  the  past,  but  the  operational  mis¬ 
sion  and  training  tasks  must  change  to  meet  MAC'S 
ever  changing  combat  role.  The  need  exists  for 
MAC  to  enhance  its  aircrew  training  by  using 
improved  training  techniques  arid  fully  exploiting 
aircrew  training  devices.  Effective  training 
devices  must  be  acqu.red  to  meet  the  command's 
trc.ning  needs. 

COMMAND  PHILOSOPHY 

The  Military  Airlift  Command's  aircrew  train¬ 
ing  goal  is  to  provide  an  efficient,  individual¬ 
ized  training  program.  To  accomplish  this  goal, 
MAC  has  acquired  aircrew  training  devices  (ATDs) 
ranging  from  basic  mockups  to  full  mission  simu¬ 
lators.  The  programs  are  structured  so  that 
students  progress  from  the  simple  devices  through 
the  most  complex  trainers  before  going  to  the 
aircraft.  Students  must  demonstrate  proficiency 
in  one  phase  prior  to  moving  to  the  next  phase. 

MAC  began  applying  Instructional  System 
Development  (ISD)  methodology  to  its  flying 
training  programs  in  Feb  72.  This  management 
teennique  has  resulted  in  the  identification  and 
procurement  of  numerous  synthetic  training 
devices  which  were  essential  to  improve  the 
quality  of  training  while  reducing  training 
costs. 

Tra i n i ng  Devices 

Aircrew  training  devices  as  described  in  AFN 
SO- 1 1  fall  into  several  categories.  MAC'*  use  of 
these  interactive  devices  is  detailed  as  follows: 

Mockups  and  Pa Ta sk  Jra i ners  are  simplified 
representations  of  afrcrafl  tysttrs  or  component 
parts.  They  are  used  to: 

a.  Introduce  and  tram  specific  procedures 

anu  trtikS- 

b.  Familiarize  and  demonstrate  sp<*  ific 
system  operations. 

c.  Practice  and  evaluate  simple  tas*s. 


Cockpit  Procedural  Trainers  are  training 
devices  which  provide  the  student  with  a  realis¬ 
tic  cockpit  environment  in  which  most  of  the  sys¬ 
tem  controls  and  indicators  operate  properly. 
These  devices  are  used  to: 

a.  Provide  initial,  continuation,  and  reme¬ 
dial  training  in  normal,  alternate,  and  selected 
emergency  procedures  which  do  not  result  in  sig¬ 
nificant  flight  dynamic  problems. 

b.  Provide  initial  and  recurring  training Jn 
ground  emergencies,  interior  preflighf,  and  aux¬ 
iliary  power  unit  operating  procedures. 

c.  Provide  loadmaster  training  in  the  opera¬ 
tions  of  cockpit  controls  where  required  (doors, 
power  units,  etc). 

d.  Provide  maintenance  crew  engine  runu)  and 
troubleshooting  procedural  training. 

Instrument  Trainers  provide  a  realistic  cock¬ 
pit  and  controls  with  an  operable  navigation  sys¬ 
tem  and  aerodynamics  package  to  allow  students  to 
perform  instrument  maneuvers.  These  devices  are 
u:>ed  to: 

a.  Teach  general  techniques  and  procedures 
for  instrument  flying. 

b.  Maintain  instrument  proficiency  or  per¬ 
sonnel  assigned  to  fly  other  than  primary  mission 
aircraft. 

c.  Augment  flight  simulator  training 
programs. 

d.  Supplement  aircraft  training  of  pilots; 
maintaining  proficiency  in  instrument  flying, 

e.  Provide  any  other  special  training  deemed 
necessary  by  local  commanders. 

Flight  Simulators  are  fully  instrumented 
advanced  training  devices  which  provide  full  sys¬ 
tem  logic.  The  operational  flight  trainers  nor¬ 
mally  do  not  have  a  visual  system  whereas  the 
weapon  system  trainers  do.  These  devices  are- 
used  to: 

a.  Fully  train  coordinated  aircrew  perform¬ 
ance  in  dynamic  scenarios  which  require  proper 
prioritization  or  management  of  cockpit  actions. 

s.  Train  normal,  alternate,  or  emergency 
procedures  which  cannot  be  trained  in  the  air¬ 
craft  due  to  lack  of  control  of  the  environment 
^crosswinds,  ceiling,  visibility,  pressure  alti¬ 
tude,  temperature,  wind  shear,  etc). 

c.  Tra’n  proficiency  and  required  alternate/ 
emergency  procedures  which  pose  a  significant 
flight  dynamic  control  problem  that  cannot  or 
should  not  be  performed  in  the  aircraft  due  to 
safety  considerations  (i.e.,  runway  aborts,  two- 
engine  approaches,  emergency  descents,  stalls, 
etc.  ). 

d.  Accerplish  required  proficiency  maneu¬ 
vers,  trai ling  tasks  or  currency  events  for  which 
the  ATD  is  ce-rti  f ied. 

?.  Conduct  aircrew  chemical  warfare 
training. 

f.  Conduct  electronic  warfare  training. 

g.  Conduct  required  flight  evaluations  t< 
iiitr  cAlci't  thtf  ATD  lci  i  i  !  it*u. 

Tra ining  Device  Use 
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In  order  to  have  the  devices  available  to 
accomplish  all  required  training,  we  have  estab¬ 
lished  the  following  priority  guidelines  for  the 
use  of  all  MAC  aircrew  training  devices: 

a.  Research  of  flight  safety  items. 

b.  Remedial  training. 

c.  Annual  refresher  training. 

d.  Special  qualification  training  (i.e.,  air 
refuel ing) . 

e.  Transition/upgrade  training  (except  in 
designated  formal  schools  where  this  is  the  high¬ 
est  priority). 

f.  Proficiency  training  (practice  and  cur¬ 
rency  ’'event"  accomplishment). 

g.  Maintenance  crew  training. 

The  guidelines  have  been  established  based  on 
historic  simulator  use  and  current  instructional 
technology.  Aircrew  training  devices  are  essen¬ 
tial  in  providing  an  effective  combat  ready  MAC 
crew  member. 

To  insure  that  sufficient  time  is  available 
t.r  maintenance  support,  the  maximum  usage  of 
older  ATDs  is  16  hours  per  day,  330  Jays  per  year 
or  5280  hours  annually.  On  newly  procured 
devices,  the  maximum  use  is  18  hours  per  day,  330 
days  per  year  or  59<U  hours  annually.  The 
increased  reliability  of  the  new  ATO's  allows 
them  to  be  used  more  than  the  older  ones. 

CURRENT  PROGRAMS 

MAC  currently  possesses  26  flight  simulators 
(operational  flight  trainers  or  weapon  system 
trainers),  12  cockpit  procedural  trainers,  two 
instrument  trainers,  and  numerous  part  task 
trainers.  These  training  devices  are  used  in  our 
C-5,  C-141,  C- 130 ,  C- 135 ,  HH-53,  and  CH-3  train¬ 
ing  programs.  Additionally,  MAC  contracts  for 
simulator  time  to  support  the  C-6,  C-i,  C-12, 
CT-39,  C-137B  and  the  C- 140  aircraft  training 
programs.  C-5  and  C-141  initial  air  refueling 
training  is  also  contracted  on  an  interim  basis. 

Initial  Training  Program 

MAC  is  committed  to  the  concept  of  a  central¬ 
ized  training  facility  to  conduct  the  majority  of 
its  aircrew  training.  This  concept  provides  a 
standardized  product  in  the  most  cost  effective 
r*eans  and  now  includes  the  basic  engineer,  air¬ 
crew  instructor,  airdrop,  air  refueling,  and  tac¬ 
tical  low  level/formation  training  as  well  as 
initial  qualification  training. 

All  MAC  funnal  schools  are  similar.  They 
consist  of  academic,  simulator,  and  flying 
phases.  The  time  at  each  school  varies  according 
to  the  aircraft  and  crew  position.  The  academic 
phase  consist*  of  programed  classroom  lectures, 
seminars,  •  ..-If -paced  instruct'on,  and  the  use  of 
syninetic  t 'diners.  The  Simulator  pha  *.*  o* 
training  inc'udes  simulators  with  motion  system 
and  visual  systems.  These  devices  allow  the  stu¬ 
dent  to  be  taught,  practice  his  skills  and  then 
be  evaluated  on  his  performance  prior  to  advanc¬ 
ing  to  the  flight  pnase.  !ne  simulators  now  in 
use  bv  MAC  are  limited  m  the  scope  of  tas»s  they 


can  train.  The  aircraft  is  used  to  complete  the 
training  and  to  allow  the  student  to  gain  self 
confidence  in  the  operation  of  an  aircraft,  stu¬ 
dents  graduate  qualified  to  perform  the  MAC  mis¬ 
sion  with  a  minimum  of  local  orientation  training 
upon  arrival  at  their  operational  unit. 

Continuation  Training  Program 

Continuation  training  is  a  program  that  con¬ 
sists  of  required  flight  currency  events,  block 
training  and  simulator  refresher/proficiency 
training.  Wing  learning  centers  provide  aircrews 
their  annual  review  of  ground  training  items 
using  slide  tape,  video  tape,  and  hands-on  train¬ 
ing  programs.  Aircrews  are  also  required  to 
maintain  their  currency  by  performing  a  specific 
number  of  currency  events  .i.e.,  nrecision 
approaches,  landings,  etc.)  within  specified  time 
periods. 

Annual  Simulator  Refresher  Training  program 
is  similar  for  all  aircraft  and  is  required  by 
Air  Force  and  MAC  regulations.  A  typical 
refresher  course  lasts  5  days  and  normally  con¬ 
sists  of  20  simulator  hour.,.  Each  day  crews  fly 
a  4-hour  mission  representative  of  a  typical  MAC 
route. 

The  fifth  day  of  the  refresher  course  is  nor¬ 
mally  a  standardization  evaluation  flight.  On 
all  missions,  the  pilots  change  sects  at  the  end 
of  the  first  2-hour  segment.  During  the  mission, 
which  is  usually  run  without  mis  on  freeze  or 
repositioning,  the  crew  is  confronted  with  vari¬ 
ous  situations  which  require  crew  coordination 
and  decision  making.  The  instructor  on  these 
missions  controls  the  pacing  of  the  mission  as 
well  as  the  content  (within  prescribed  guide¬ 
lines).  He  guides  the  student,  when  necessary, 
through  the  situation  with  immediate  feedback, 
alternate  actions,  and  then  follows  up  with  a 
post  mission  debriefing. 

During  the  week-long  course,  all  aircraft 
systems  are  covered.  The  student  must  be  able  to 
operate  the  system,  recognize  malfunctions,  ana¬ 
lyze  the  problem,  and  take  the  appropriate  cor¬ 
rective  actions.  Numerous  realistic  malfunctions 
and  emergencies  are  presented.  Additionally 
varying  instrument  situations,  including  category 
!I  ILS  training  in  tne  C-141,  are  included,  This 
training  not  only  includes  minimum  visibility  and 
ceiling  approaches,  but  other  weather  phenomena 
such  as  low  level  wind  shear. 

In  addition  to  the  actual  time  spent  in  the 
training  device,  each  rn ssioh  includes  a  2-ho*r 
premission  brief inn.  This  briefing  is  controlled 
by  the  instructor,  but  normal),  is  more  of  a  dis¬ 
cussion  than  briefing.  It  covers  the  systems  and 
procedures  to  be  presented  on  that  mission.  Fol¬ 
lowing  The  mi^'irtn.  i  ijehripfinq  nt  the  maior 
points  of  the  mission  are  covered  by  the  instruc¬ 
tor.  At,  this  point  the  s^dents  have  a  further 
opportunity  to  discuss  systems,  procedures,  and 
alternate  courses  of  action  for  varying 
situations. 
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Simulator  Proficiency  Training  program  is  two 
to  three  days  in  length  and  provides  8  to  12 
hours  of  simulator  time.  It  concentrates  on  mis¬ 
sion  training  and  skill  maintenance.  For  stra¬ 
tegic  airlift,  the  course  is  tailored  to  exten¬ 
sive  instrument  training.  The  tactical  airlift 
course  will  use  the  fidelity  of  the  new  C-130 
operational  flight  trainer  (OFT)  to  train  station 
keeping  and  airdrop  procedures.  The  training 
potential  of  the  single  C-130  weapon  system 
trainer  (WST)  is  currently  being  explored  to  pro¬ 
vide  visual  formation,  low  level  navigation  and 
visual  airdrop  training. 

In  the  mid  70 ' s ,  MAC  decided  that  some  maneu¬ 
vers  performed  in  the  simulator  could  be  substi¬ 
tuted  for  aircraft  required  currency  events.  A 
subjective  certification  of  each  device  was  con¬ 
ducted.  It  was  determined  that  aircraft  required 
approaches  could  be  replaced  by  instrument 
approaches  performed  in  the  simulator.  During 
this  same  time  period,  credit  was  given  for  all 
simulator  time  logged  in  computing  upgrade 
requirements. 

SIMCERT 

in  uic  mi  J  *  dilution  cut?  mi 

Staff,  directed  that  each  command  establish  a 
program  to  insure  that  training  devices  being 
procured  were  used  effectively.  The  Air  Force 
simulator  certification  (SIMCERT)  program  was 
established  to  implement  chis  directive.  The 
purpose  of  the  SIMCERT  program  is  to  tnsure  that: 

a.  The  training  capability  of  each  aircrew 
tnining  device  is  defined  in  order  to  be  effec¬ 
tively  integrated  into  the  proper  aircrew  train¬ 
ing  program. 

b.  All  ATDs  receive  update  modifications  as 
requi red. 

c.  All  ATDs  continually  perform  according  t) 
set  stan^rds. (5) 

The  MAC  SIMCERT  program  meets  the  stated 
criteria  by  conducting  annual  certi fication  test¬ 
ing  of  each  ATD.  This  testing  uses  the  contrac¬ 
tor  developed  acceptance  test  procedure  as  its 
baseline.  In  addition  to  testing  the  ATD,  the 
SIMCERT  team  also  evaluates  the  training  course¬ 
ware  to  insure  the  capabilities  of  the  training 
device  are  being  effectively  used.  As  new  equip¬ 
ment  is  installed  on  the  aircraft,  the  SIMCERT 
team  evaluates  the  training  requirement  and 
recommends  modification  requirements  for  the 
ATDs. 

A  SIMCERT  team  has  been  established  for  C - I 3U 
at  Ltttle  Rock  AFB,  Arkansas.  The  C-S  and 
C-14!  SIMCERT  team  is  located  at  Altus  AFB, 
Oklahoma.  Each  team  is  Composed  of  aircrew 
instructors  and  maintenance  personnel.  In  addi¬ 
tion  to  their  other  duties,  the  SlMcERI  team  is 
the  command's  design  team  'or  acquisition,  mod¬ 
ification,  development,  and  design  reviews.  This 
approach  includes  the  users  in  tne  design  process 
at  an  early  stage  so  that  they  r.ay  help  identify 
or  farther  define  the  users'  training  require¬ 
ments.  Another  benefit  could  be  ire  early 


identification  of  trainer  deficiencies  and  to 
provide  user  interface  when  program  or  scope 
changes  are  anticipated. 


Simulation  Training  Studies 


There  are  several  ongoing  studies  being  con¬ 
ducted  on  MAC  simulators.  A  transfer  of  training 
study  is  attempting  to  determine  the  value  of  the 
C-130  General  Electric  visual  system  for  low 
level  navigation  and  tactical  training.  A  per¬ 
formance  measurement  system  is  being  installed  on 
tne  C-5  simulators  at  Altus  to  determine  its  use¬ 
fulness  to  the  instructors.  A  proposed  study  of 
skills  maintenance  is  of  vital  concern  to  MAC. 
As  an  interim  measure,  MAC  is  developing  a  study 
to  determine  the  effectiveness  of  the  Air  Refuel¬ 
ing  part  task  trainer  and  its  skills  maintenance 
potential . 


BENEFITS  OF  SIMULATION 


Through  the  years,  MAC  has  greatly  benefited 
from  simulation  in  aircrew  training.  The  major 
benefit  derived  is  increased  proficiency  cf  r cw 
members.  They  are  able  to  develop  and  practice 
their  cockpit  management  skills  when  placed  in  a 
task  loaded  situation.  They  have  gamed 
increased  systems  knowledge  and  the  simulation 
allows  the  crews  to  better  understand  the  air¬ 
craft's  limitations  and  evaluate  mission  capa¬ 
bilities  in  a  degraded  mode  of  operation.  The 
capability  to  contro.  the  aircraft,  systems, 
environmental  conditions,  aircraft  location  and 
other  traffic  has  greatly  enhanced  the  probabil¬ 
ity  of  MAC  aircrews  adapting  to  an  unusual  or 
challenging  situation. 


A  second  benefit  o.‘  simulation  is  the  ability 
to  reallocate  the  training  cr  mission  resources. 
Using  simulators  to  perform  tasks  that  are  time 
consuming,  require  extensive  preparation  or  spe¬ 
cific  resources  allows  valuable  flying  time  tc  be 
reallocated.  This  time  can  then  be  used  to  con¬ 
duct  additional  training  or  satisfy  mission 
requi rements. 


A  third  benefit  is  that  of  flight  safety. 
There  are  numerous  risky  maneuvers  that  are  per¬ 
formed  in  training  that  place  the  aircraft  near 
the  edge  of  its  perfornidr.ee  ervelone.  Usinu  the 
simulator  to  initially  train  and  practice  them 
maneuvers  on  a  routine  basis  will  reduce  the  risk 
to  a  crew  and  their  aircraft.  By  using  tne  simu¬ 
lator,  the  conditions  can  be  control  lt  d,  the  stu¬ 
dent  receives  practice  and  can  be  evaluated  at  no 
risk,  to  tr.ose  involved. 


Another  benefit  of  simulation  is  flying  Hour 
avoidance.  Although  secondary  m  importance,  it's 
often  viewed  as  the  primary  benefit  by  many, 
especially  in  today's  economy.  Simulation  can 
save  valuable  aircraft  uuu'S,  fuel,  and  operating 
costs.  In  1983  MAC  estimates  that  the  following 
additional  hours  would  h«>  required  to  maintain 
the  same  level  of  aircrew  proficiency  if  simula¬ 
tion  were  not  available: 


C-5 

342c 

nOui’S 

C-141 

24356 

hours 

C-130 

1913 

hours 

HH-53 

714 

hours 

CH-3 

1087 

hours 

C- 135 

276 

hours 

1 

37774 

hours 

These  hours  represent  approximately  70  mil¬ 
lion  gallons  of  jet  fuel.  The  total  dollar  sav¬ 
ings  wuulfl  ue  appr ua  turn lely  $23..  ffillllCm. 

Wnen  all  C-130  OFTs  are  fully  operational  we 
estimate  an  additional  flying  hour  avoidance  of 
6385  hours.  This  will  save  4.8  million  gallons 
of  jet  fuel  and  a  total  of  approximately  SI  1.1 
mil  1 i on  per  year. 

In  terms  of  the  learning  process,  simulation 
has  many  benefits.  The  benefits  described  above 
are  the  primary  benefits  to  MAC.  The  overall 
bottom  line  is  that  simulation  allows  MAC  to  pro¬ 
vide  highly  qualified  crew  members  trained  to 
perform  the  MAC  mission. 

FUTURE  PROGRAMS 

MAC!s  goal  is  to  develop  a  total  instruc¬ 
tional  system  that  will  provide  aircrew  personnel 
a  continuuP’  of  training  from  entry  into  the 
career  field  through  to  a  fully  qualified 
instructor  status.  In  the  future,  crew  members 
should  find  a  well  rounded,  complementary  program 
in  the  initial,  continuation  and  upgrade  training 
programs.  Most  training  items  will  be  thoroughly 
learned  on  the  ground  and  validated  in  the  air¬ 
craft.  The  training  system  should  include  learn¬ 
ing  centers,  an  effective  family  of  aircrew 
training  devices,  and  aircraft. 

To  meet  tnis  goal,  future  aircrew  training 
device  acquisitions  will  follow  the  philosophy  of 
acquiring  a  family  of  training  devices  to  allow 
training  in  a  "building  block1'  fashion. 

The  newest  program  is  ine  acquisition  of 
seven  C-5/C-141  air  refueling  part-task  trainers 
(ARPTT).  The  ARPTT  will  be  a  generic  cockpit  to 
train  the  pilots  in  air  refueling.  The  specific 
task  will  be  from  the  pre-contact  (approximately 
500  feet  behind  the  tanker)  position  through 
hock-up  and  disconnect  *'rom  the  tanker.  This 
system  must  include  a  wide  f ield-of-view,  high 
fidelity  visual  system.  The  trainer  will  be  used 
for  both  initial  crew  qualification  and  '-'freshet 
training  for  qualified  crews.  An  ARP7 I  that 
meets  MAC’S  requirements  has  the  potential  tor 
great  savings  and  improvements  in  aircrew  train¬ 
ing.  The  savings  are  assoc iaied  with  the  goal  of 
an  all  air  refueling  force  and  its  r elated  train¬ 
ing  requirement  .  ’here  is  an  insufficient  num¬ 
ber  of  tankers  to  uppurt  the  currency  require¬ 
ments  ana  by  using  the  APP1T,  MAC  will  be  able  to 
maintain  a  lull/  qualified  air  -'efueling  force. 

Both  the  H-oO  helicopter  program  and  the  next 
transport  program  have  requi t ements  fur  a  family 
of  training  devices:  procedure  trainers. 


operational  flight  trainers  (simulator  without 
visual)  and  weapon  system  trainers  (full  mission 
simulator  with  visual). 

MAC  also  has  plans  to  refurbish  the  existing 
C-5  flight  simulators  in  conjunction  with  the 
procurement  of  the  C-53  ATD's.  Currently,  the 
C-141  and  C- 135  simulators  are  on  contract  for 
complete  refurbishment.  These  programs  shou^ 
produce  trainers  that  will  be  logistical ly  sup¬ 
portable  for  years  to  come. 

The  Command  is  also  determining  the  useful¬ 
ness  of  Special  Function  Part  Task  Trainers  to 
train  and  update  aircrew's  knowledge  and  profi¬ 
ciency  levels.  These  are  interactive  computer 
based  training  systems  that  can  be  used  to  simu¬ 
late  new  aircraft  hardware  or  equipment  modifica¬ 
tions.  In  addition,  the  systems  can  be  used  for 
student  management,  administrative  support  and 
data  gathering. 

Line-oriented  flight  trairing  (LOFT),  used 
by  many  airlines,  can  be  very  beneficial  to  our 
training  system.  MAC'S  programs  have  aiway-  been 
K5cnH  unon  a  realistic  mission  nrofi  !e  but  tb°y 
continue  to  be  very  task  saturated.  The  profi¬ 
ciency  programs  are  being  revised  to  incorporate 
the  LOFT  concept.  The  missions  will  be  struc¬ 
tured  to  require  the  aircrew  to  make  decisions 
and  take  appropriate  actions  based  upon  mission 
requirements,  aircraft  operations  and  existing 
conditions.  The  instructors  will  control  the 
conditions,  monitor  progress  and  evaluate  the 
training  needs  of  the  students.  Instructors 
normally  hold  their  comments  and  critiques  till 
the  mission  debriefing.  Additionally,  we  envi¬ 
sion  more  specialized  training  for  each  crew 
position  tailored  to  the  individual's  experience. 

The  future  of  simulation  in  the  MAC  programs 
is  briqht.  Plans  are  to  continue  the  use  of  sim¬ 
ulation  with  an  increased  emphasis  on  individual  - 
iz^d  training. 

PROBLEMS 

MAC'S  aircrew  training  device  programs  are 
not  without  problems.  Some  of  these  problems 
nave  been  in  the  consnand  since  1953,  and  solu¬ 
tions  st’ 11  are  not  readily  available. 

I  tie  majO''  problem  facing  the  day-to-day 
’’’dining  operation  is  inadequate  manpower. 
Although  increased  instructor  authorizations  have 
been  identified  as  a  valid  requirement ,  funding 
tor  these  authorizations  ha  :ioi  bee*  4 pp roved. 
This  requires  the  use  of  "on-man"  ’rst’uctors 
from  airlift  squadrons  to  fill  the  snort f a P, 
which  is  detrimental  to  the  squadron's  training 
program. 

The  lengthy  acquisition  process  f  or  »(t* 
training  devices  presents  a  second  p*-obien. 
Training  requirements  are  identified,  evtluated 
end  reeds  are  established  through  A.r  Force  chan¬ 
nels.  Ter:  years  may  ,  m  before  the  device 
actually  arrives.  A  prime  example  of  tm \  the 
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new  C  - 1 30  flight  simulator'.  The  original 
requirement  was  written  in  1971.  The  acceptance 
of  the  first  of  10  devices  was  in  1981. 

The  problem  also  exists  when  modifying  or 
upgrading  training  devices.  Some  progress  has 
been  made  towards  resolving  this  lengthy  proce¬ 
dure.  In  the  past  the  simulator  conf iguraticn 
lagged  that  of  the  actual  aircraft.  Changes  in 
Air  Force  regulations  and  improvements  within  the 
Ogden  Air  Logistics  Center  (simulator  depot)  and 
MAC  have  brought  the  simulator  configuration 
closer  to  that  of  the  aircraft.  Expedience  indi¬ 
cates  that  close  cooperation  between  the  user 
(MAC)  and  the  depot  (Ogden)  significantly  ^educes 
this  problem. 

Defining  user  requirements  in  terms  under¬ 
standable  to  the  acquisition  agency  has  long  been 
a  problem.  The  translation  of  user  requirements 
into  an  engineering  specification  produces  varied 
results.  As  far  back  as  1953  when  MATS  vas 
attempting  to  acquire  a  C-118  flight  simulator, 
"certain  differences  of  opinion"  between  MATS  and 
the  Air  Research  and  Development  Command  occur¬ 
red.  (6)  The  sam^  is  today.  A  better  means 
of  communication  still  needs  to  be  devised. 

Problems  also  occur  after  a  contract  is 
awarded  for  a  new  device.  As  the  design  pro¬ 
gresses,  certain  tradeoffs  must  be  made.  Many 
factors  affect  these  tradeoff  decision*.  Often 
neither  the  user  nor  the  acquisition  agency  is 
involved  until  the  decisions  are  fi-al.  This 
often  results  in  schedule  slips,  cost  overruns, 
or  decreased  capability.  The  user  must  wait 
until  the  device  has  completed  acceptance  testing 
before  hn  can  develop  a  training  program  utilis¬ 
ing  the  device's  capabilities.  The  continuing 
uncertainty  if  delivery  dates  and  training 
capabilities  continue  to  be  of  concern  to  MAC. 

CONCLUSION 

The  Military  Airlift  Command  is  proud  of  itt 
aircrew  training  programs.  By  effectively  using 
a  family  of  ATOs ,  imulation  has  proved  to  be  «? 
niort  effective  training  tool.  With  it,  MAC  is 
able  to  produce  quality  aircrew  members  beginning 
with  the  initial  qualification  programs  and  then 
maintaining  their  readiness  in  the  operational 


environment.  Simulation  qreatly  enhances  this 
capability  and  results  in  significant  savings  to 
the  Air  Force.  Heretofore,  MAC  has  not  had  ade¬ 
quate  ATOs  to  train  such  tasks  as  air  refueling, 
instrument  and  visudl  formation,  tactical  low 
level  navigation,  airdrop  and  asrault  operations. 
The  anticipated  advances  in  technology  and  the 
continued  use  of  simulation  with  an  increased 
emphasis  on  individualized  training  will  allow 
MAC  to  effectively  train  these  and  other  tasks. 
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ABSTRAC  i 


VTXTS  is  t h:*  totally  integrated  trailing  system  designed  to  satisfy  the  U.S.  Navy's 
Undergraduate  Jet  Flight  Training  requirements  for  the  1990s  ana  beyond.  The  acquisition  of 
ihe  VTXTS  to  replace  the  current  Intermediate  and  Advanced  Strike  Flight  Training  programs 
of  the  Naval  Air  Training  Commana  will  climax  several  years  of  intense  effort  on  the  part  of 
the  Na  al  Air  Systems  Commands,  McDonnell  Douglas  Corporation  and  the  ultimate  user,  the 
Naval  Air  Training  Command.  VTXTS  is  one  of  the  first  major  defense  acquisitions  wherein 
the  entire  training  system  is  being  designed,  developed  and  produced  by  a  single 
contractor.  This  process  is  affording  the  Navy  the  opportunity  to  explore  innovative  ideas 
and  advances  in  technology  in  all  areas  of  the  training  system. 

-In  this  paper  we  will  demonstrate  how  the  operational  and  instructional  inputs  by  the  Chief 
of  Naval  Education  and  Training  (CNET)  and  its  major  functional  command,  the  Chief  of  Naval 
Air  Training  (CNATRA),  have  been  and  will  continue  to  be  significant  factors  in  ensuring 
that  VTXTS  will  be  responsive  to  the  needs  of  the  Navy  tor  a  state-of-the-art  jet  training 


system  through  the  next  twenty  years., 
BACKGROUND 

The  Naval  Air  Training  Command  :ia:  been 
training  highly  skilled  and  qualified  aviators 
for  the  fleet  for  decades.  In  recent  years  the 
flying  skills  and  tactical  knowledge  required  of 
th?  newly  designated  aviator  as  he  moves  to  his 
fleet  assignment  have  advanced  to  a  level 
undreamed  of  in  years  past.  The  unique  and 
demanding  requiiements  of  carrier  aviation  have 
always  been  reflected  in  the  training 
methodologies  of  the  Naval  Air  Training  Command, 
and  the  training  system  has  continually  been 
revised  and  updated  to  reflect  the  current  fleet 
needs.  The  structure  of  the  Navy's  Undergrad¬ 
uate  Pilot  Training  Program  has  also  changed  to 
meet  the  current  needs  of  each  era.  The  Naval 
Air  Training  Command  utilizes  separate  training 
"pipelines”  for  different  aircraft  missions. 
With  this  pipeline  concept,  separate  training 
systems  are  established  for  three  general 
aircraft  mission  types:  tactical  or  strike 
aircraft,  mnltiengine  aircraft  and  helicopters. 

Student  Naval  Aviators  (SNA)  for  the  Navy, 
Marines  and  the  Coast  Guard  enter  the  Naval  Air 
Training  Command  from  a  variety  of  sources 
including  the  U.S.  Naval  Academy,  The  Coast 
Gut  rd  Academy,  College  RO'C  programs  and  both 
Navy  and  Marine  Corps  Officer  Candidate 
Schools.  They  are  trailed  within  the  Naval 
Integrated  Flight  Training  System  (NiFTS)  which 
begins  for  ail  students  with  Preflight  at  the 


Laval  Aviation  Schools  Command  in  Pensacola, 
Florida.  After  Pre  t light  the  Student  Naval 
Aviators  begin  their  flight  training  in  a  common 
Primary  phase  flying  the  T-34C  aircraft.  At  the 
completion  of  Primary  flight  training,  students 
are  selected  for  one  of  three  Intermediate 
training  pipelines,  strike,  maritime  (multi- 
engine)  or  helicopter.  Students  in  the  strike 
pipeline  fly  the  North  American  T-2C  twin  engine 
jet  for  Intermediate  phase  and  the  McDonnell 
Douglas  TA-4J  for  Advanced.  The  VTXTS  will 
replace  these  two  aircraft  with  a  single 
aircraft  and  a  totally  integrated  training 
system  for  the  strike  pipeline. 

CONCEPT 

VTXTS  is  a  total  training  system,  composed 
of  a  jet  trainer  aircraft,  a  suite  of  simula¬ 
tors,  academics,  and  a  computerized  training 
management  system.  These  four  elements  are 
integrated  into  mutually  supportive  building 
blocks  which  enable  the  Student  Naval  Aviator  to 
meet  the  training  requirements  in  the  most 
effective  and  efficient  manner.  The  VTX  training 
system  is  the  functional  integration  of  these 
building  blocks  withi/i  a  working  process.  This 
integration  is  shown  schematically  in  Figure  1. 
The  ongoing  evaluation  and  coordination  of  this 
functional  integration  by  CNET  as  well  as  by 
CNATRA  will  ensure  that  the  final  product  of 
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this  system,  the  designated  Naval  Aviator,  will 
meet  the  needs  of  the  fleet.  It  will  also 
ensure  tha*  fleet  feedback  is  promptly  evaluated 
and  appropriate  revisions  to  the  training  system 
are  effected.  VTXTS  is  one  of  the  first  major 
defense  acquisitions  wherein  the  entire  training 
system  is  being  designed,  developed  and  produced 
by  a  single  contractor  and  procured  by  the  Navy 
as  a  fully  integrated  system  and  program 
package.  This  process  is  affording  the  Navy  the 
opportunity  to  explore  innovative  ide^s  and 
advances  in  technology  in  all  the  elements  of 
the  training  system,  not  just  in  the  area  of 
aircraft  design.  The  acquisition  process  is 
also  providing  an  exceptional  opportunity  for 
the  immediate  using  command,  CNATRA,  to  provide 
continual  and  ongoing  inputs  throughout  the 
design  and  development  cycles.  The  fact  that 
the  erl  product  of  this  acquisition  is  a 
craining  system,  not  a  weapons  system,  has 
facilitated  the  exploration  of  new  and  inno¬ 
vative  ideas  in  training  equipment,  metnod- 
ologies,  and  advanced  training  technologies.  A 
key  task  of  the  prime  contractor  is  to  include 
the  operational  inputs  of  the  using  command  in 
the  integrated  design  engineering  development 
proc^'-s.  The  end  objective  is  a  working  system 
that  provides  the  required  training  levels  and 
rates  over  a  ? 0  year  program  at  the  lowest  cost 
to  train. 

Any  discussion  of  user  input  to  the  VTXTS 
should  include  a  description  of  the  relationship 
between  CNET  and  CNATRA.  Since  1972  CNET  has 
been  responsible  for  all  formal  Navy  training. 
CNATRA  is  one  of  several  functional  commands 
under  CNET  and  draws  on  the  assets  cf  the  CNET 
staff  for  many  areas  of  technical  and  training 
support.  Some  of  CNET's  other  functional 
subordinates  include  tne  Naval  Training  Equip¬ 
ment  Center  (NTEC),  the  Navy  Recruit  Training 
Command,  the  Chief  of  Naval  Technical  Training 
(CNATECHTRA)  and  the  NROTC  Command. 

The  formulation  of  the  VTXTS  concept  began 
in  the  mid-1970s  when  CNET  directed  CNATRA  to 
conduct  the  initial  Undergraduate  Pilot  Training 
(UPT)  Task  Analysis.  The  purpose  of  tnat  UPT 
Task  Analysis  was  two-fold:  first,  to  verify  the 
existing  training  procedures  and  curricula,  and 
identify  any  deficiencies,  and  second,  to  apply 
these  findings  to  vhe  training  program  in 
achieving  the  required  modifications  and 
improvements.  The  results  of  this  task  analysis 
were  published  in  the  Phase  I  and  Phase  11  Task 
Analysis  Reports.  This  UPT  Task  Analysis  initi¬ 
ated  the  participation  of  the  CNATRA  staff  in 
the  acquisition  process  for  the  VTXTS.  It  has 
been  this  participation  at  each  step  in  the 
conceptualization  and  design  process  that  will 
help  to  ensure  that  the  VTXTS  will  meet  the 
needs  of  the  using  commands. 

The  thread  of  CNATRA's  active  partici¬ 
pation  in  the  VTXTS  p; ocess  continued  when  the 
Chief  of  Naval  Operations  (CNO)  requested 
industry  participation  in  conceptual  design 
studies  of  an  Advanced  Navy  Jet  Trainer.  A 
competitive  solicitation  was  conducted  to  obtain 
ideas,  approaches  and  conceptual  designs  'or 
this  aircraft.  The  Naval  Air  Development  Center 
(NADC)  contracted  with  four  aircraft  manufac¬ 


turers,  McDonnell  Douglas  Corporation,  General 
Dynamics  Corporation,  Northrop  Corporation  and 
Vought  Corporation  for  the  conceptual  design 
studies.  Briefings  and  dialogue  with  the  four 
contractors  established  a  definite  need  to 
address  all  elements  of  this  total  training 
system  rather  than  just  the  aircraft.  Both  CNET 
and  CNATRA  had  been  very  active  in  the  early 
efforts  leading  up  to  these  studies.  In 
addition  to  the  UPT  Task  Analyses  that  CNATRA 
had  conducted,  the  Naval  Training  Equipment 
Center  had  completed  the  Training  Situation 
Analysis  (TSA)  specifically  for  the  VTXTS,  in 
which  various  training  media  required  for  a 
state-of-the-art  training  system  were  ad¬ 
dressed.  The  baseline  requirements  of  the 
system  used  by  the  four  contractors  in  their 
technology  studies  were  based  on  discussions 
with  the  Naval  Air  Training  Command  staff  and 
instructor  pilots.  VTXTS  workshops  were  con¬ 
ducted  at  CNATRA  in  preparation  for  the  technol¬ 
ogy  base  studies  and  the  contractors  had  access 
to  tne  Task  Analysis  Reports,  the  TSA  and  the 
then-current  NIFTS  curriculum.  At  the  comple¬ 
tion  of  these  technology  base  studies,  an 
analysis  team  was  established  to  review  and 
summarize  the  results  of  the  stales.  CNET  and 
CNATRA  were  again  heavily  involved  in  the  review 
process.  Members  of  the  CNE.r  and  CNATRA  staffs, 
as  well  as  Subject  Matter  Experts  (SME's)  from 
Training  Wings  Two  and  Three  were  included  on 
the  analysis/review  committee.  The  review 
committee  assessed  the  study  data  in  various 
area,  including  methodology,  concept  options, 
effectiveness  measurement  and  cost. 

In  March  1979  the  Mission  Element  Need 
Statement  (MENS)  for  the  VTXTS  was  signed  by  the 
Secretary  of  Defense  signalling  the  formal 
initiation  of  the  VTXTS  program.  Under  0MB 
Advisory  Circular  A-1^9,  the  Mission  Element 
Need  Statement  formally  establishes  the  need  for 
the  system  involved  and  signals  Department  of 
Defense  concurrence.  The  signing  of  the  MENS  is 
Milestone  zero  in  a  specified  series  of  deci¬ 
sions  by  the  Defense  System  Acquisition  Review 
Council  (DSARC).  Following  the  issuance  of  the 
MENS,  a  Request  for  Quotation  (RFQ)  was  issueo 
fur  the  Alternative  Systems  Exploration  studies 
in  December  1979.  Once  again,  CNATRA  partici¬ 
pation  in  the  acquisition  process  was  evident. 
CNATRA  assembled  a  team  composed  of  CNATRA  VTXTS 
Special  Projects  members,  representatives  of  the 
training  wing  staffs  and  instructor  pilots  from 
the  training  squadrons.  In  conjunction  with  the 
CNET  Educational  Specialists,  this  team  devel¬ 
oped  the  Terminal  Learning  Objectives  (71.0's) 
tor  the  VTXTS.  The  87  TLO's  provided  the  means 
to  define  the  specific  goals  of  t m*  VTX  training 
system.  The  TLO's  describe  flying  skills  that 

the  student  pilot  will  be  expected  t,o  demon¬ 

strate  prior  to  the  completion  of  the  VTXTS 
curriculum.  CNATRA  staff  members  also  provided 
operational  inputs  to  the  Naval  Air  Systems 

Command  (NAVA1R)  for  the  Constraints  and 
Guidelines  section  of  the  Request  for 
Quotation.  As  a  result  of  this  competitive 
solicitation,  six  contractors  were  awarded 
contracts  for  Alternative  Systems  Exploration 
V ASE )  studies.  These  six-month  studies  enabled 
the  contractors  to  complete  in-depth  analyses 
and  trade  studies  on  the  varied  aspects  of 


the  training  system  requirements.  Within  the 
limitations  of  the  MENS,  the  RFQ  Constraints  and 
Guidelines,  and  the  TLO's,  each  contractor  was 
able  to  propose  its  own  training  system  includ¬ 
ing  recommendations  for  the  allocation  and 
arrangement  of  the  system  elements. 

As  the  acquisition  process  was  originally 
established  under  the  provisions  of  0MB  Advisory 
Circular  A-109,  The  Naval  Air  Systems  Command 
was  to  evaluate  the  ASE  studies  and  select  one 
or  more  contractors  for  continuation  into  the 
following  acquisition  phase.  Demonstration/ 
Validation  (DEM/VAL).  The  DEM/VAL  phase  was  to 
be  a  risk  reduction  process  in  which  the 

contractor  would  develop  selected  portions  of 
the  propoied  training  system  tor  demonstration 
and  validation  cf  the  concept.  In  November  1981 
the  McDonnell  Douglas/British  Aerospace  proposal 
was  chosen  in  the  source  selection  as  the 

winning  entry.  With  the  narrowing  of  the 
selection  process  to  a  single  contractor  team, 
the  concept  of  a  DEM/VAL  phase  has  changed  to  a 
Pre-Full  Scale  Development  phase,  Pre-FSD.  The 
Pre-FSD  phase  will  be  followed  by  the  Full-Scale 
Development  and  ultimately  the  Production  phase 
of  the  VTXTS  acquisition.  As  the  VTXTS  acquis¬ 
ition  process  continues,  a  cooperative  working 
relationship  will  be  established  between  the 
contractor,  McDonnell  Douglas  Corporation,  and 
the  using  commands,  CNET  and  CNATRA.  The 

beginnings  of  this  working  relationship  that 
have  evolved  after  the  source  selection  are 
help.ng  to  ensure  adequate  user  input  to  the 

training  system  in  its  design  phase.  In  the 
future,  the  benefits  of  this  early  user  input  to 
the  development  process  will  be  even  greater. 
By  the  early  integration  of  the  end  user's 
requirements  into  the  training  system  develop¬ 
ment  process,  the  resultant  system  will  reflect 
more  accurately  the  real  needs  and  requirements 
of  the  users. 

PROJECTIONS 

Early  in  the  VTXTS  acquisition  cycle, 
CNATRA,  as  the  primary  using  command,  estab¬ 
lished  a  VTXTS  Special  Project  team.  This  group 
is  composed  of  CNATRA  staff  personnel  with 
extensive  backgrounds  in  all  areas  of  the  Naval 
Air  Training  Command  operations  including  flight 
operations,  simulator  instruction,  academics, 
maintenance,  facilities  management  cr*id  general 
military  training.  The  experience  of  this 
group,  coupled  with  CNET's  staff  o?  instruc¬ 
tional  technologists,  enabled  the  using  commands 
directly  involved  with  the  VTXTS  to  provide  the 
operational  input  that  is  so  vital  to  the 
acquisition  of  a  totally  integrated  system 

McDonnell  Douglas  Training  Systems 
Engineers  recognize  the  importance  o4  this 
operational  expertise  to  the  success  of  the 
VTXTS.  Tne  timely  and  proper  utilization  of 
that  expertise  as  the  system  acquisition  prog¬ 
resses  will  be  a  key  factor  in  ensuring  that  the 
VTX  training  system  will  be  responsive  to  the 
real  world  needs  of  the  training  command  users 
in  the  late  1980's  beyond. 

CNET/CNATRA  input  to  the  development  pro¬ 
cess  is  expected  to  be  especially  valuable  in 


the  areas  of  curriculum  outline  refinement, 
final  definition  of  the  simulator  functional 
equipments  and  operational  inputs  to  the 
raining  Management  System  (TMS).  The  final 
integrated  academic,  simulator  and  flight  cur¬ 
riculum  will  ue  a  reflection  of  the  total  VTX 
system  concept.  It  is  essential  that  the  ever- 
changing  operational  requirements  as  identified 
by  CNATRA  staff  as  well  as  training  wing  and 
training  squadron  personnel  be  integrated  into 
the  VIXTS  curriculum  to  ensure  that  it  is  both 
operationally  and  1 nstructional ly  sound.  The 
Training  Management  System  (TMS)  represents  an¬ 
other  area  where  the  user  input  will  be  espe¬ 
cially  important.  In  the  current  NIFTS  the 
individu*1  squadron's  schedules  officer  is  a  key 
to  the  successful  flow  of  students  through  the 
curriculum.  The  schedules  officer  assembles 
information  on  stucent  and  instructor  availabil¬ 
ity,  aircraft  availability  academic  and  simula¬ 
tor  requirements,  and  a  myriad  of  scheduling 
directives  into  a  daily  flight  schedule.  The 
operational  expertise  of  key  personnel  such  as 
these  schedules  officers  will  be  vital  to  the 
implementation  of  the  Training  Management  System 
into  the  training  command  structure. 

CONCLUSIONS 

The  operational  environment  of  the  Naval 
Air  Training  Command  is  unique  and  demanding. 
It  is  one  of  the  best  military  p’lot  training 
programs  in  the  world.  It  is  dynamic  and 

ever-changing  in  response  to  the  needs  of  the 
fleet.  The  tact  that  the  Naval  Air  Training 
Command  has  maintained  this  pre-eminent  position 
is  a  tribute  to  the  CNATRA  staff,  the  Training 
Wing  staffs,  the  training  squadron  personnel  and 
the  support  personnel  a«.  the  training  bases. 
The  challenges  presented  to  tne  prime  contrac¬ 
tor,  McDonnell  Douglas,  in  developing  the  VTXTS, 
integrating  the  elements  of  the  system  and 

implementing  the  system  into  the  Navi'l  Air 

Training  Command  are  indeed  formidable.  The 
ta>k  will  require  a  closely  coordinated  effort 
between  the  contractor  and  the  using  command  tor 
identification  of  real-time  operational  require¬ 
ments  and  the  integration  of  those  requirements 
into  the  training  system.  The  ground  work  for 
this  close  cc'-;  erat  ion  between  user  and  contrac¬ 
tor  has  been  laid.  As  the  development  effort 
moves  ahead  during  the  FSD  phase,  this  ground 
worx  will  form  the  basis  tor  a  cooperative 
working  relationship  between  user  and  contractor 
that  is  vital  to  the  ultimate  success  of  the 
training  system.  To  date  all  contacts  between 
contractor  and  user  have  been  closely  coor¬ 
dinated  and  controlled  Dy  the  VTXlb  acquisition 
manager ,  the  Naval  Air  bystems  Command.  NAV AIR 
will  continue  to  function  in  this  important 
position  as  coordinator  between  user  and 
contractor. 

The  entire  VlXTb  concept  of  a  total 
training  system  with  all  of  its  elements-- 
aircraft,  simulators,  academics  and  training 
management  system,  designed  and  developed  by  a 
single  contractor--  is  new.  Tne  challenges 
facing  the  training  command  today  are  also  new 
and  ever-changirg.  The  cost  of  training 
resources  has  risen  dramatically  and  the 
continued  availability  of  current  assets  has 
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become  difficult  to  maintain.  VTXTS  will  enable 
the  Navy  to  reduce  the  cost  to  train  each 
student  by  providing  a  fuel -efficient  training 
aircraft,  improved  simulator  capability,  an 
interactive  computer-assisted  academic  system 
and  an  automated  training  management  system. 
The  ever-rising  cost  of  fuel  is  a  significant 
factor  in  the  total  cost  to  train.  The  new  VTX 
aircraft  with  its  fuel -efficient  engine  and  its 
increased  reliability  over  the  current  strike 
pipeline  aircraft  will  provide  a  major  reduction 
in  training  costs.  The  state-of-the-art  VTXTS 
simulator  suite  and  the  interactive  Computer 
Assisted  Instruction  (CAI)  system  allow  the 
number  of  required  flight  hours  to  be  reduced 
substantially,  providing  yet  another  major  cost 
reduction.  Many  of  the  peripheral  admin¬ 
istrative  demands  placed  on  flight  instructors 
by  the  current  NIFTS  will  be  automated  in  VTXTS 
by  the  Training  Management  System.  Instructors, 
relieved  of  many  of  those  administrative, 
bookkeeping  ype  responsibilities,  will  have 
more  products  1  time  available  for  actual  direct 
instructor  to  student  interface  within  the 
instructional  er.ironment,  ensuring  an  even  more 
well-trainjd  Naval  Aviator  than  ever  before. 
The  level  of  sophistication  and  advanced  tech¬ 
nology  m  the  Navy's  modern  fleet  aircraft 
requires  ever-increasing  flying  skills  and 
systems  management  skills  on  the  part  of  the 
newly  designated  Naval  Aviator.  VTXTS  will 
provide  him  with  basis  ot  those  skills. 

The  need  for  VTXTS  is  clear,  and  the  need 
for  close  integration  of  the  CNET  and  CNATRA 
experience  in  training  command  operations  with 
the  broad  b*se  of  McDonnell  Douglas  expertise  in 
aircraft  and  systems  engineering  is  equally 
evident.  It  will  be  the  coope-ative  working 
relationship  between  user  and  contractor  which 
will  result  in  the  development  of  a  viable 
training  system  capable  of  producing  quality 
Naval  Aviators  for  the  fleet  at  a  lower  cost- 
to-train  than  ever  before.  The  VTXTS  will  meet 
the  Navy's  reed  for  a  quality  training  system  in 
the  Naval  Air  Training  Command.  The  VTXTS 
trained  Naval  Aviator  will  be  even  more  well 
prepared  to  transition  to  the  fleet  aircraft  of 
the  next  two  decades  than  his  predecessors 
were.  He  will  be  ready  to  meet  the  challenges 
and  rewards  of  a  career  in  earner  aviation  with 
the  United  States  Navy. 
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ABSTRACT 

c u t offli r S  P<lPnr  addresst‘s  trainin£  device  logistics  support  problems  encountered  by  both  the 

nrohTl  rrLl  l  r  af°r'  .Tt  describes  sPecific  *”PPort  area  problems,  and  secondary 
problems  created  by  implementing  short  term  solutions.  Many  of  the  problems  were  initially 

cause ^  - y  a  rapidly  changing  technology,  and  an  environment  beyond  the  control  of  the  con-' 
n  Pities.  As  the  paper  traces  through  attempted  solutions,  it  endeavors  to  extrapolate 

et  torts  into  n  _ _  _  r  ,  r 
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past  and  present  efforts  into  a  projected  support  concept  for  the  future. 


INTRODUCTION 

Over  the  past  thirty  years,  the  armed  forces 
of  the  United  States  of  America  have  made  great 
strides  in  the  development  of  new  and  unique 
weapons  systems.  With  the  advent  of  spa~e  age 
technology,  these  systems  have  become  more  complex 
with  future  growth  potentials  that  are  virtually 
unlimited.  Along  with  this  trend,  the  costs 
associated  with  owning  and  operating  these  systems 
has  increased  enormously.  Training  of  operator 
and  support  personnel  has  become  a  major  problem, 
and  logistics  support  concepts  have  become  as 
complicated,  and  as  expensive  as  the  systems  them¬ 
selves.  The  hirh  cost  of  operating  these  systems 
his  made  their  use  for  training  purposes  cost- 
prohibitive.  As  an  alternative,  the  use  of 
training  devices  to  conduct  training  has  become 
even  more  cost  effective  when  compared  to  con¬ 
ducting  training  on  the  actual  operational  system. 
Continuing  as  a  natural  outgrowth  of  this  trend 
and  spurred  on  by  the  rapid  advances  in  state  of 
the  art  technology,  there  has  been  an  impetus  to 
incorporate  increasing  realism  into  the  training 
device,  producing  simulators  that  closely  repli¬ 
cate  the  operational  system  and  its  environment. 
Consequently,  the  simulation  device  becomes  not 
only  a  complex  procurement  exercise  but  also  a 
major  support  problem.  The  scope  of  the  problem, 
and  the  costs  involved,  are  second  only  to  those 
encountered  in  supporting  the  operational  system. 


A 


It  is  axiomatic  that  th£  solutions  implemented  to 
solve  today's  problems  will  dictate  the  support 
concepts  of  the  future^ 

The  paper  draws  upon  the  experiences  of  the 
authors  primarily  in  the  areas  of  aircraft,  simu¬ 
lation  programs  for  the  military.  The  application 
of  statements  made  and  conclusions  drawn  to  other 
types  of  training  devices  similar  in  complexity, 
are  considered  valid. 

CURRENT  MAJOR  PROBLEM  AREAS 

It  is  obvious  that  an  attempt  to  analyze  all 
the  problems  concerning  the  logistics  support  of 
training  devices  would  require  an  in-depth  study 
and  would  result  in  volumes  of  data.  This  paper 
identifies  some  of  the  current  major  problem  areas 
and  describes  them  without  attempting  to  sequen¬ 
tially  rank  them  in  order  of  importance.  These 
problem  areas  are  all  inter-related  and  are  gener¬ 
ally  driven  by  the  acquisition  process.  Several 
areas  noted  in  the  paper  were  addressed  in-depth 
in  last  years  proceedings.  Each  discussion  will 
encompass  the  problem,  related  cost  impacts,  past 
solutions,  and  anticipated  current  results.  The 
problem  aroas  selected  for  discussion  will  be 
addressed  individually,  and  are  as  follows: 


To  keep  pace  with  the  increasing  complexity, 
full  scale  logistics  support  programs  have  been 
developed  to  meet  the  customers*  requirements.  As 
the  logistics  base  has  expanded  to  meet  the  unique 
support  requirements  of  each  customer,  the  costs 
of  simulators,  including  design  documentation, 
training,  spares  support,  test  equipment  and 
maintenance  data  have  increased.  In  attempting 
to  minimize  acquisition  costs  and  meet  production 
schedules,  the  trends  exhibited  by  both  contractor  . 
and  customer  have  been  concentrated  toward 
increased  simulation  capability,  placing  logistics 
support  secondary.  This  situation  is  not  unique 
to  the  training  device  community. 

Recognizing  the  evolutionary  process  that 
weapon  systems  have  undergone  over  the  past  thirty 
years,  this  paper  endeavors  to  provide  a  few 
observations  on  the  corresponding  evolution  of 
training  devices,  and  training  device  support  con¬ 
cepts.  These  observations  become  apparent  when 
the  major  problem  areas,  both  past  and  present, are 
examined  and  how  the  attempts  at  solving  these 
problems  have  influenced  current  support  concepts 


•  Device  Sophistication 

•  Commercial  Off-The-Shelf  Equipment 

•  Reliability,  Maintainability  and 
Avai lability 

•  Spares  Support 

•  The  Changing  Maintenance  Concept 

•  Personnel  and  Training 

•  Data 

Device  Sophistication 

The  training  device  has  evolved  from  a 
rudimentary  training'  aid  to  be  used  in  perfecting 
procedures  and  maintaining  basic  proficiency  to  a 
complex,  highly  versatile  piece  of  hardware  that 
is  capable  of  provid ing_de tai led  operational  and 
tactics  training  in  various  programmable  real 
world  scenarios . 
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•'s  *'  ’ 1  c  1,1 1  1  ;l,,fi  tactics  b,- ft»i«i'  inon*  soph  is  t  i  - 
i-lt''l,  Llu-v  dictate  ovifn  more  sophisticated  train- 
' !D'  (i‘’v i  cos  tor  the  various  armed  services.  In 
‘  ni-  •i,’>  t  liov  f  roi| in* n L  1  v  .I'iv.ini’c  state-of-the-art. 

' n  !iu’  (-'aso  or  flight  simulators,  the  device  must 
ilup  1  i  ca  to  t!u>  i  p.tor  i  or  cockpit  section  in  appear- 
anco  a:n!  provide  spec  i  f  i  ed  performance  character- 
i>tics  identical  to  that  o;  the  actual  aircraft. 

In  addition,  tin-  dev  i  co  must  be  capable  of  simu¬ 
lating  the  mu  1 1.  i -mi  ss  i  on  training,  scenarios 
required  by  the  military. 

T'>  achieve  a  simulation  design  that  faithfully 
reproduces  the  operational  aircraft  and  provides 
for  an  effective  training  system,  the  first  task 
encountered  is  to  clearly  specify  the  extent  of 
sinmlat ion  required  and  the  training  requirements 
or  goals,  i.o.,  customer's  planned  utilisation  of 
the  device  as  an  instructional  system.  These  tasks 
should  receive  maximum  effort  and  should  be  defin¬ 
able  given  that  Liu-  operational  system  has  achieved 
a  reasonable  level  ot  design  maturity.  The  second 
task  requires  the  contractor  to  accurately  inter¬ 
pret  those  requirements.  Not  withstanding  the 
technical  and  training,  aspects  of  simulator  design, 
the  maintenance  or  logistics  support  concept  must 
also  be  clearly  stated  by  the  customer  to  the 
extent  necessary,  and  interpreted  accurately  by 
the  contractor.  Obviously,  close  liaison  between 
customer  and  contractor  is  required  throughout  the 
program's  Life  cycle. 

The  increased  sophistication  and  requirements 
Lor  multiple  training  scenarios  are  driving  the 
need  tor  large  capacity,  high  speed  computer 
systems.  This  in  turn  requires  tin*  development  of 
high  spued  data  interfaces  to  provide  the  real  time 
s imu la t i on- requ i red  to  achieve  the  realism  desired. 
Computer  systems  as  we L t  as  many  of  the  high  speed 
data  interfaces,  are  of  a  commercial  nature.  Also 
considered  as  commercial  hardware  (catalogued-! toms ) 
for  purposes  of  this  example,  are  visual  and  graphic 
displav  systems  and  peripheral  hardware  to  the  com¬ 
puter  system  such  as  line  printers,  CRTs,  disc  and 
magnetic  tape  equipment.  Thus,  to  meet  design  and 
delivery  criteria,  an  ever  increasing  amount  of  off- 
the-shelf  equipment  is  included  in  the  design.  From 
a  parts  count  viewpoint.  60-70  percent  of  the  simu¬ 
lation  hardware  is  resident  within  the  commercial 
s vs  tens . 

Comma rc ial  Off-The-Shelf  Equ i pmen t 

The  use  of  commercial  off-the-shelf  equipment 
items  in  the  manufacture  of  training  devices  has 
become  common-prnct ice  in  the  training  device 
industry.  This  practice  evolved  as  a  result  of 
efforts  by  both  government  and  industry  to  minimize 
acquisition  costs,  and  provide  realistic  and 
obtainable  delivery  schedules  in  response  to  the 
terms  and  conditions  of  government  contracts.  A 
positive  outcome  of  this  was, and  continues  to  be, 
the  development  of  a  market  place  for  various  com¬ 
mercial  systems. 

Usually,  commercially  available  high  speed 
computer  systems  comprise  the  heart  of  a  training 

device.  Further,  these  systems  many  times  are _ 

state-of-the-art.  The  reasons  for  this  approach 
are  quite  evident:  first,  there  are  no  MIL-STD 
computer  systems  available  that,  meet  the  demanding 
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simulation  requirements.  Development  of  a  MIL-STD 
system  would  be  cost-prohibitive.  Additionally, 
the  government  procurement  process  being  what  it 
is,  state-of-the-art  advances  might  well  make  the 
system  obsolete  prior  to  it  becoming  operational. 
Secondly,  significant  ist  reductions  are  realized 
as  K&I)  costs  for  commercial  computer  systems  are 
shared  by  all  users.  Without  the  lengthy  K&I) 
process,  it  is  much  easier  for  training  device 
manufacturers  to  meet  required  device  delivery 
schedules.  Finally,  the  use  of  commercially 
available  computer  systems  eliminates  some  of  the 
unknown  factors  associated  with  R&D.  Reliability 
and  maintainability  characteristics  many  times  have 
been  proven  through  demonstration  or  field  results. 
Quite  often,  the  selected  system  is  already  inte¬ 
grated  into  current  customer  inventories  which 
reduces  the  scope  oi  the  logistics  support  problem. 

Hi e  bottom  line  is  that  the  training  device  environ¬ 
ment,  especially  in  flight  simulators,  is  basically 
benign  and  a  MIL-STD  computer  is  not  required. 

At  first  glance,  it  appears  that  significant 
cost  reductions  are  realized  through  the  use  of 
off-the-shelf  computer  systems.  In  most  cases, 
this  approach  is  used  advantageously.  There  are,,.-  - 
however,  some  risks  associated  with  this  method. 
Generally,  a  commercial  computer  manufacturer  will 
produce  a  specific  model  line  for  a  limited  period 
of  time.  He  cannot  afford  to  exclusively  cater  to 
the  training  device  industry  because  the  market  is 
too  small.  Ho  must  produce  for  the  commercial 
market.  To  remain  competitive  in  the  market  place, 
he  must  keep  abreast  with  changing  technology. 

With  the  advent  of  subminiaturization  of  compo¬ 
nents,  improved  packaging  techniques,  and  the  wide 
availability  of  low  cost  memory  modules,  the  com¬ 
mercial  computer  industry  is  growing  at  an  unpre¬ 
cedented  rate.  The  life  span  of  a  product  line 
seldom  exceeds  five  years.  By  contrast,  the  pro¬ 
grammed  life  span  of  a  training  device  is  seldom 
under  20  years.  Similarly  with  the  comments  made 
about  MIL-STD  computers,  the  time  elapsed  before 
the  government  procurement  process  places  a  system 
in  the  field  causes  the  commercial  system  to  be 
potentially  obsolete.  This  is  generally  true  with 
any  government  procurement.  One  major  problem  is 
that  training  devices  are  not  procured  in  quanti¬ 
ties  comparable  to  other  military  hardware. 
Therefore,  logistics  support  of  spare  parts  manu¬ 
factured  by  commercial  vendors  is  not  likely  to 
remain  viable  over  the  programmed  life  span  of 
the  training  device.  Obtaining  guarantees  from 
vendors  to  assure  suppor tab i li ty  and  configuration 
control  over  the  trainer's  life  cycle  may  be  cost 
prohibitive  or  impossible  to  obtain.  Many  manu¬ 
facturers  do, however,  provide  "downward  compat¬ 
ibility"  of  parts  for  superceded  systems.  There 
is  some  risk  here  in  maintaining  both  in  hardware 
and  software,  the  total  integrity  of  the  original 
sys  tern. 

In  terms  of  operating  hours  and  trainer  sortie 
rates,  utilization  of  training  devices  far  exceed 
that  of  the  operational  aircraft.  Although  the 
environment  is  totally  different,  complexity  of 
aircraft  training  devices,  approaches  and  often 
exceeds  that  of  the  aircraft.  Obviously,  the 
logistics  support  concept  for  a  training  device 
must  he  approached  with  a  perspective  different 
from  that  of  a  major  operational  system.  An  air¬ 
craft  manufacturer,  or  any  manufacturer  with  large 
quantities  of  equipment  in  the  field,  must  provide 
a  support  capability  for  that  equipment. 
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8 imi  I e r  1  y  ,  ,1  trii!  niii};  device  mnnu  faeLurer  must 
■  ilso  provide  support.  The  vendors  he  buys  from 
ore  not  so  obligated.  With,  current  trends  towards 
depot,  level  maintenance  for  computer  components, 
tiie  vendor  in  many  oases  becomes  the  depot  level 
repair  or  exchange  lacility.  Equipment  turn¬ 
around  times  and  replenishment  spares  become  a 
probl  ’tn  t: . i a f  increases  in  direct  proportion  to 
tiie  Lile-span  ot  the  device.  In  many  cases,  the 
onl\  apparent  solution  to  the  problem  is  a  cos  1 1 v 
computer  update  to  the  exist  ins  training  device, 
because  the  or  ip,  inn1  computer  has  become  increas¬ 
ingly  tmsuppor tab  1 e .  Specifications  can  be 
written  to  eliminate  all  but  a  particular  vendor 
without  actually  referring  to  a  product  line. 

'I’ll  i  s  is  doin'  because  of  customer  satisfaction 
with  a  particular  product  or  in  an  attempt  to  use 
spares  ;r  other  support  assets  that  mav  already 
no  in  his  inventory.  A  potential  pitfall  here  is 
that  vendors .  in  providing  the  depot  level  repair 
or  exchange  sorvee,  tend  to  update  their  compo¬ 
nents  to  the  latest  design  revision  level.  The 
part  number  of  the  component  remains  essentially 
the  same  however,  the  customer  cannot  differenti¬ 
ate  between  revision  Keels.  Consequently,  the 
component  may  or  may  not  be  fully  interchangeable 
with  the  systems  in  the  fit' Id  and  the  customer  may 
have  unknowingly  degraded  his  support  capability. 

1’erhaps  the  most  significant  drawback 
resulting  from  the  use  of  commercial  computer 
systems  is  that  the  training  device  industry 
cannot  place  a  direct  demand  on  computer  manu¬ 
facturers  to  ensure  that  their  products  wilL 
continue  to  meet  the  requirements  for  sophisti¬ 
cated  simulation.  The  potential  problems  of  off- 
the-shelf  computers  may  be  analogous  to  problems 
of  supporting  older  tube-type  electronic  systems. 

The  customer  may,  at  some  point  in  time,  have 
to  pay  the  price  to  bridge  the  technology  gap. 
Reference  can  be  made  to  those  draft  specifications 
circulated  for  industry  review  and  comment  whore 
the  customer  is  told  bv  industry:  "we  -'anr.ot 
get  there  from  here  with  today's  state-of-the-art." 
Attempts  to  bridge  the  state-of-the-art  gap  may 
result  in  excessive  expenditure  of  R&D  funds  with 
the  potential  for  poor  equipment  performance  in 
the  i  iold  in  that  the  hardware  has  unknown  and 
unproven  reliability  characteristics.  This  impacts 
not  onlv  the  operational  aspects  but  suppor t ab L1 i ty 
as  well'. 

Reliability,  Maintainab ill ty  and  Avai lability 

In  general,  most  contracts  call  out 
Reliability  and  Maintainability  plans  and  demon¬ 
strations  to  prescribed  LcveLs  of  performance. 

The  customer  apparently  has  two  distinct  concerns. 
First,  from  the  operations  standpoint,  he  is  con¬ 
cerned  w.th  equipment  or  system  availability,  ancl 
does  not  care  about  the  combination  of  MTBF  and 
MTTR  required  to  achieve  that  level  of  avail¬ 
ability.  Secondly,  he  is  concerned  with  the  cost 
of  supporting  the  item.  From  the  second  concern 
steins  the  rigid  reliability  and  maintainability 
requirements.  As  graphically  shown  in  Figure  1, 
once  the  MTTR  is  under  five  hours  with  an  MTBF 
above  one  hundred  hours,  increases  in  MTBF  add 
little  increase  in  marginal  availability.  Thus, 
in  respect  to  inherent  availability,  the  efforts 
in  reliability  arc  misdirected. 
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Figure  1 

Up  to  this  point  a  theoretical  inherent  avail¬ 
ability  has  been  described,  but  the  actual  concern 
should  be  wi th  achieved  availability.  The  elements 
of  supply  times,  operator  and  maintenance  induced 
failures,  and  preventive  maintenance  times  must  be 
added  to  the  downtime  portion  of  the  equation. 

From  the  operators  position,  the  downtime  caused  by 
scheduled  maintenance,  operator  induced  failures, 
and  downtime  caused  by  nonrespons i vo  supply  systems 
mast  be  reduced.  These  causes  for  downtime  produce 
a  far  greater  los.,  of  availability  than  the  inherent 
fa i  lures  . 

Log  is t i cans  and  support  personnel  should  be 
less  concerned  with  inherent  system  reliability, 
and  focus  their  attention  on  the  areas  of  supply 
support,  scheduled  maintenance,  and  the  inherent 
relia'  ility  of  high  cost  or  scarce  components. 

System  failure  data  provided  in  accordance  with 
MIL-STD  785  has  little  or  no  relationship  to 
achieved  availability  nor  to  cost  of  supporting  a 
system.  More  attention,  therefore, must  be  placed 
upon  high  cost  or  scarce  components  during  early 
design  reviews.  This  is  where  reliability  efforts 
pay  oLf. 

Spares  Su pport 

With  availability  in  mind,  rapid  repair  of 
the  simulation  device  has  been  driven  to  remove  and 
replace  at  the  organizational  level.  This  repair 
is  limited  to  the  removal  and  replacement  of  parts 
and  minor  overhaul  of  nonelectronic  parts.  Main¬ 
tenance  at  the  intermediate  level  has  diminished 
because  most  mult i- Layered  PCBs  are  routed  to  a 
military  supply  depot  for  repair  or  disposition. 

Many  of  these  boards  are  then  routed  to  a  special 
repair  activity,  normally  a  contractor  or  vendor 
plant.  The  boards  are  then" repaired  and  returned 
through  the  same  channels.  This  circuitous  routing 
may  consume  many  months.  This  lengthy  process  is 
expensive  in  both  administrative  handling  and  in 
terms  of  additional  spares  to  fill  the  pipeline. 

It  is  a  generally  accepted  fact  that  pipelines  have 
a  habit  of  "eating"  spares. 

In  many  simulator  programs  the  government  and 
contractors  have  reduced  the  pipeline  time  by  direct 
contractor  or  vendor  support  programs  of  spares. 

On  Navy  programs,  Repair  of  Repairable  contracts 
are- established  during  the  interim  support  and 
follow-on  phases  to  expedite  s i te-to-vendor  repairs. 
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This  is  an  approach  to  support  which  reduces  cost, 
increases  device  availability,  and  increases  over¬ 
all  pi*  r  fo  ruianco  .  Ic  should  be  realized  however, 
that  this  procedure  further  degrades  government 
organic  support  capabilities  even  though  costs  may 
be  the  overriding  factor. 

What  is  observed  in  these  types  of  procedures 
is  a  do  facto  change  in  the  maintenance  concept. 

A  formal  recognition  of  this  change  must  be  con¬ 
sidered  during  the  provisioning  phase  of  the 
acquisition  process,  due  to  the  increased  pipeline 
turnaround  times.  A  broader  acknowledgement  and 
appreciation  of  the  actual  maintenance  concept 
should  begin  a  process  in  achievement  of  cost- 
etlective  approaches  to  training  device  support. 

The  Changing  .-!a i  ntenance  Concept 

Today  as  in  the  past,  systems  are  being 
designed  for  maximum  self  sufficiency  at  the  base 
level  with  increased  emphasis  on  the  organizational 
level  of  maintenance.  The  extensive  use  of  Built- 
in-Test  (BIT),  Bui  l  t'-i  n-Tost-Equi  pment  (BITE), 
daily  readiness  checks  and  demonstrations  and 
associated  diagnostic  routines  have  enabled  the 
organizational  level  technician  to  fault  isolate 
to  a  much  lower  level  than  in  the  past.  Equipment 
is  being  designed  tor  ease  of  maintenance  with 
modularity  in  mind  to  enhance  repair  times  and 
optimize  device  availability. 

Historically,  the  armed  services  have  employed 
a  three  level  maintenance  concept:  organizational , 
intermediate  and  depot.  The  organizational  level 
has  generally  consisted  of  removal  and  replacement 
o!  major  components  "on-equipment"  with  some  bit/ 
piece-part  repair  when  applicable.  The  major  pur¬ 
pose  b-'Lng  to  return  the  end  item  to  an  operational 
condition  in  a  minimum  amount  of  time.  The  inter¬ 
mediate  level  has  consisted  of  "of f-cquipmont" 
repairs  (to  those  major  components  removed  at  the 
"0"  level)  using  the  same  remove  and  replace  theory 
at  the  modular  level.  The  main  purpose  here  is  to 
return  a  serviceable  major  component  back  into  the 
supply  system  to  support  the  "0"  level  of  mainten¬ 
ance.  The  depot  level  has  included  those  repair 
capabilities  at  both  "0"  and  "I"  level  with  the 
additional  capability  to  perform  major  equipment 
overhaul.  The  intent  being  to  perform  those  repairs 
or  major  overhauls  that  are  extensive,  time  con¬ 
suming  and/or  exceed  "0"  or  "I"  level  capabilities. 

Although  the  basic  concepts  have  remained 
the  same  over  the  years,  device  sophistication, 
technology  and  a  quest  for  device  availability 
have  dictated  procedural  changes  for  the  main¬ 
tenance  of  certain  types  of  equipment.  These 
changes  have  altered  the  basic  maintenance  con¬ 
cepts  . 

One  change,  and  probably  most  significant, 
has  been  the  trend  to  relegate  the  repair  of  high 
technology  printed  circuit  hoards  to  the  depot 
level  of  maintenance.  The  reasons  for  this  move 
are  many  and  will  not  bo  discussed  in  detail. 
However,  cost  is  a  major  contributor  when  examined 
from  the  standpoint  of  the  additional  systems, 
personnel  and  training  required  to  provide  this 
repair  capability  for  intermediate  level  main¬ 
tenance  at  multiple  locations.  The  impact  when 
applied  to  the  maintenance  of  flight  simulators 
lias  been  significant.  Tn  maintaining  flight 


simulators,  unlike  the  aircraft,  PCU's  are  gener¬ 
ally  replaced  on  equipment  at  the  "0"  level  of 
maintenance.  As  stated  previously,  n  large  per¬ 
centage  of  a  flight  simulator  is  high  technology 
commercial  equipment  and  the  respective  F’CB  *  s  are 
generally  returned  to  the  depot  for  repair.  As  a 
result,  the  intermediate  level  maintenance  function 
has  become  less  effective  in  that  it  does  not  pro¬ 
duct!  a  serviceable  component  to  support  the  "O" 
level  maintenance  of  high  technology  equipment 
i  terns . 

This  fact  must  be  recognized  by  the  customer 
when  he  drafts  his  specifications  and  data  require¬ 
ments  prior  to  advertising  for  proposals.  He  has 
recognized  this  situation  in  the  personnel  and 
training  area,  and  has  implemented  programs  to 
address  it.  He  must  also  address  this  situation 
during  the  acquisition  process. 

Personnel  and  Training 

The  end  of  the  military  draft  system  has 
compounded  the  logistics  support  problem.  Coupled 
with  a  shrinking  pool  of  potential  recruits,  it 
becomes  increasingly  difficult  for  the  military  to 
fulfill  enlistment  quotas.  The  current  recession 
and  high  unemployment  have  provided  some  temporary 
relief,  but  the  problem  still  exists.  Under  the 
present  system,  the  manpower  situation  cannot  be 
expected  to  improve  in  the  future,  due  to  the 
demographics  of  the  situation.  Perhaps  the  most 
painful  point  to  make  is  that,  in  most  cases,  the 
military  recruits  and  trains  a  technician  and 
industry  induces  him  to  separate  after  his  first 
term  of  service.  The  proliferation  of  home  com¬ 
puters,  video  games,  video  recorders,  and  other 
electronic  products  will  also  compete  for  services 
of  the  trained  technician.  The  customer  finds  it 
difficult  to  maintain  organically,  all  of  the 
sophisticated  hardware  procured. 

To  further  aggravate  the  situation,  many 
career  military  personnel  have  dual  skill  quali¬ 
fications.  As  the  simulator  technician  alternates 
between  jobs,  his  level  of  proficiency  is  somewhat 
degraded,  depending  on  the  length  of  time  he  is 
away  from  the  training  device  environment.  A 
case  in  point  is  the  recent  decision  by  the  Navy 
making  simulator  technicians  (TD  rating)  eligible 
for  sea  duty. 

There  are  no  immediate jmswers  to  these 
problems.  Assuming  solutions  were  available  and 
implemented  today,  it  would  be  some  time  before 
the  benefits  could  be  realized.  The  military 
personnel  problem  is  certainly  not  a  new  one. 

It  has  been  recognized  in  the  past,  and  a  number 
of  measures  have  been  implemented  to  correct  it. 
This  paper  does  not  intend  to  discuss  the  success 
of  these  measures;  it  only  points  out  thac  the 
problem  exists,  and  that  it  has  an  impact  on  pro¬ 
viding  the  required  logistics  support  for  *  •i.iiag 
devices . 

The  second  part  of  the  personnel  probiei.,  .s 
training.  Some  assumptions  can  be  made  based  on 
observations.  The  military  approach  to  training 
is  to  ensure  that  those  trained  personnel  will 
remain  in  the  military  for  a  sufficient  length  of 
time  to  provide  a  return  on  the  training  invest¬ 
ment.  In  the  past,  the  military  provided  in-depth 
training  in  the  highly  technical  maintenance 
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Ck,rVl'r  ,)uri”K  tli  i  <i  time  a  largo  number  of 

Uainod  personnel  elected  to  make  the  military  a 
career.  Over  the  past  few  years,  we  have  observed 
some  radical  departures  from  this  approach  as 
addressed  in  The  Changing  Maintenance  Cpncopt. 

Cone  rally,  m i 1 i t  a  ry  train i ng  prog  rams  have 
been  reduced  to  the  level  appropriate  to  the 
remove  ami  replace  maintenance  concept.  This  has 
enabled  the  military  to  train  v  w  recruits  to  be 
productive  in  a  minimum  amount  of  timo.  With  this 
approach,  lengthy  training  is  net  required. 

Although  detailed  training  is  certainly  available, 
it  is  generally  provided  only  to  career  military 
personnel  or  civilian  technicians.  Perhaps  the 
most  neteable  observation  to  be  made  here  is  that 
the  military  is  slowly  losing  or  eroding  one  of 
their  most  valuable  resources,  the  highly  trained 
and  skilled  technician. 

1  know  the  valie*  of  training  programs  and 
that  cost  •  effect ive  training  must  be  commensurate 
with  the  tasks  to  be  performed.  In  training 
devices,  maintenance  personnel  require  a  level  of 
training  higher  than  that  to  satisfy  a  remove  am! 
replace  concept  only.  The  maintenance  of  training 
devices  requires  highly  skilled  technicians  capable 
o!  troubleshooting  ami  making  repairs  at  both  the 
organizational  and  in  termer*  i  a  te  levels  of  main¬ 
tenance,  Thus  tie  level  of  training  must  reflect 
the  complexity  of  modern  simulators. 

Growth  in  the  electronics  industry  is  approx¬ 
imately  17  percent  per  annum. ^ 1 ^  The  demand  for 
skilled  electronic  technicians  by  industry  will 
aggravate  the  problem  the  military  faces  in  re¬ 
taining  skilled  technicians.  An  increasing  demand 
lor  a  diminishing  resource  is  foreseeable. 


Onto 


The  data  requirements  contained  in  specifi¬ 
cations  and  contracts  are  a  major  contributor  to 
the  overall  acquisition  cost.  In  many  instances 
it  may  be  as  high  as  thirty  percent  of  the  total. 
Since  training  devices  are  typically  procured  in 
small  quantities,  many  of  the  data  requirements  are 
of  marginal  value  when  measured  against  their__cost. 
Since  design  is  based  upon  the  specification  and 
fixed  early  in  the  program,  later  efforts  in  such 
trade  studies  as  Life  Cycle  Cost  (LCC)  are  for  all 
practical  purposes  a  paperwork  exercise.  The  same 
may  be  said  of  many  aspects  of  Logistics  Support 
Analysis  (LSA),  because  a  very  large  portion  of 
the  training  device  consists  of  off-the-shelf 
hardware  and  designs  are  essentially  fixed.  How¬ 
ever,  some  value  of  LCC  and  LSA  can  he  realized 
through  their  use  in  the  equipment  selection  pro¬ 
cess;  greater  value  if  significant  production 
quantities  are  anticipated.  With  a  preponderence 
of  commercial  components  in  a  training  device,  any  * 
measure  of  system  reliability  becomes  a  meaningless 
goal.  Reliability,  maintainability  and  avail¬ 
ability  should  bo  considered  as  an  entire  package 
in  respect  to  the  end  item  being  procured. 

The  relative  merit  of  comparing  acquisition 
cost  and  O&S  costs  can  he,  and  has  been,  disected 
many  times  through  many  schools  of  thought.  This 
paper  does  not  want  to  belabor  the  issue,  only  to 
describe  ways  in  which  some  of  those  costs  could 
be  reduced.  If  the  trend  towards  full  contractor 
support  continues,  then  some  of  the  high  cost  items 
required  by  contract  become  non-cost  effective. 


Mil  Spec  technical  manuals ,  documentation,  life 
cycle  cost  programs  and  programs  to  achieve  avail¬ 
ability  have  great  importance  if  the  device  is  to 
be  supported  totally  with  organic  resources.  If 
the  device  is  being  supported  by  a  contractor, 
thev  lose  much  of  their  potential  value  in  light 
of  their  acquisition  costs. 

.  Requests  for  Proposal  for  recent  simulator 
acquisitions  places  an  excessive  burden  on  the 
contractors  tc  provide  a  plethora  of  reports  and 
data  items  early  in  the  contract  period.  In  many 
cases,  this  requirement  is  of  marginal  value,  a 
premature  requirement,  or  a  mere  "square  filling" 
exercise.  To  procure  a  cost  effective  flight 
simulator ,  the  government  must  first  clearly  iden¬ 
tify  what  is  wanted,  then  determine  how  it  will  be 
supported,  and  finally,  close  cooperation  must 
exist  between  the  contractor  and  government  to 
ensure  that  the  product  meets  the  requirement. 

WHAT  DOES  IT  ALL  MEAN? 

The  problems  faced  in  supporting  a  training 
device  are  analogous  to  the  problems  faced  in 
designing  the  system.  Each  element  of  support 
may  be  compared  to  a  simulation  module  in  the 
trainer.  Addressed  in  isolation,  a  single  sup¬ 
port  area  or  simulation  module  is  a  relatively 
straight  forward  task.  However,  neither  of  these 
tasks  exist  in  isolation;  they  must  be  made  to 
work  in  harmony  among  themselves  and  with  the 
rest  of  the  system. 

The  contract  specification,  increasing  device 
sophistication,  use  of  commercial  equipment, 
availability,  personnel  and  training,  documen¬ 
tation  and  spates  have  been  discussed.  The  same 
problems  that  have  challenged  the  logistics  com¬ 
munity  for  years  still  exist,  nothing  new,  includ¬ 
ing  the  use  of  commercial  equipment  and  the  impacts 
of  technological  change.  What  is  new  are  the 
changes  in  logistics  support  concepts  that  have 
occurred  over  the  years  in  attempting  to  minimize 
cost  and  solve  support  problems  caused  by  the 
rapid  growth  in  technology.  Both  customer  and 
contractor  have  been  driven  to  investigate  and 
implement  new  al ternatiues .  In  doing  so,  systems 
are  being  designed  for  maximum  self-sufficiency 
at  the  base  level  with  increased  emphasis  on  the 
organizational  level  of  maintenance.  Spares  are 
normally  stored  at  the  device  site.  These  measures, 
driven  by  the  environment,  have  resulted  in  a 
major  change  in  the  military's  basic  maintenance 
concept  as  it  is  applied  to  training  devices. 
Technology  and  cost  have  driven  the  services  to 
relegate  the  repair  of  printed  circuit  hoards  at 
the  depot  level  of  maintenance. 

The  authors  realize  that  there  are  redundant 
statements  made  within  and  between  the  identified 
problem  areas.  However,  it  is  just  as  difficult  to 
address  a  particular  support  area  in  isolation  as 
it  is  to  solve  overall  support  problems  by  only 
sub-optimizing  the  individual  support  elements. 

To  achieve  optimum  results  in  both  cost  and  per¬ 
formance,  all  support  element’s  including  Lhe 
operational  aspects  must  he  addressed  as  an  inte¬ 
grated  effort. 
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EVOLVING  SUPPORT  CONCEPTS 

,',Q,)_NT?C\CT I  .‘ft;  AND  ACQUISITION  STRAW: < ! V *  3)  - 
Oh-  cent  racr  trig  community  needs  to  employ”" 
ma  »>\  different  t  oo  1  s  and  t  e  eh  n  i  q  ue  s  i  n 
acquiring  items  necessary  for  the  effi- 
vMMit  op,>  rat  i  on  of  our  forces  In  both 
peace  and  war.  tncroased  weapon  system 
costs  and  prices  of  goody  and  services, 
i  etjui  i  ey  the  development  oi  approaches 
‘li:suro  readiness  through  the  most 
efficient  methods  of  contracting  and 
acquisition.  Therefore,  this  thrust 
area  consists  of  the  development  of 
new  and  innovntiv*  techniques  for  im¬ 
proving  the  overall  contracting  and 
acquisition  process." 
l'TS2  Air  For Ci*  Logistics  Research  and 
S  t.  mi  i  es  Program. 

ni*“re  are  several  approaches  to  reducing  the 
acquisition  and  support  costs  of  training  devices. 
Inese  approaches  range  from  a  simple  modification 
oi  an  existing  procurement  practice  such  as  a 
sole-source  contract,  to  a  major  departure  from  an 
existing  maximum  self-sufficiency  policy,  to  pur¬ 
chasing,  only  the  training  service.  This  latter 
change  in  policy  would  rely  on  the  contractor  to 
build,  maintain,  and  operate  the  training  device. 

.ho  government  agency  would  purchase  onlv  the  use 
of  the  device. 

The  easiest  procedural  change  for  the  govern¬ 
ment  to  make  is  to  add  increased  emphasis  in 
tailoring  its  data  requirements  to  the  end  item 
being  procured.  In  the  area  of  training  devices, 
particular ly  flight  simulators,  a  large  portion  of 
C h e  cust ome r ' s  p r o c u romon t  consists  of  e onne r c i a  1 
equipment  and  GET.  With  this  fact  in  mind,  it 
appears  that  some  I.CC  studies,  LSAs  ,  and  reli¬ 
ability  requirements  may  he  misdirected  efforts. 

Vhese  requirements,  and  perhaps  others,  could  be 
mollified  or  eliminated  from  certain  RFPs  and  con¬ 
tracts.  This  would  simply  involve  the  tailoring 
of  specifications  to  meet  the  intended  method  of 
support.  If  the  government  chooses  to  use  con¬ 
tract  field  engineering  services  rather  than  pro¬ 
vide  its  own  maintenance,  it  does  not  require  Mil 
Spec  maintenance  data;  operator  or  maintenance 
training  and  training  data;  an  LSA;  nor  Mil  Spec 
engineering  data.  This  approach  can  provide  the 
potential  for  significant  savings  in  the  overall 
ac q u is i t i on  cos t s . 

Contract  field  engineering  services  can  be 
carried  a  step  further  by  having  the  contractor 
also  provide  supply  support.  As  previously  dis¬ 
cussed,  the  cost  of  a  military  supply  system 
includes  more  than  just  the  administrative  costs 
of  handling  and  storing  spare  parts.  It  also 
includes  the  costs  associated  with  the  increased 
spares  required  for  the  pipeline.  In  training 
devices,  this  pipeline  has  become  longer  due  to 
the  decreased  effectiveness  of  intermediate  level 
maintenance.  Although  a  trade  study  of  the 
relative  merits  of  contractor  spares  support 
versus  government  supply  support  has  not  been 
conducted  to  the  knowledge  of  the  authors,  such 
a  study  would  certainly  yield  interesting  results. 

The  concept  of  contractor  support  is  not  in 
itself  a  radical  departure  from  existing  practice. 

The  armed  services  have  in  some  cases  contracted 
for  base  operating  support  services,  including 


janitorial  services,  food  services,  medical  ser¬ 
vices,  and  so  on.  The  Air  Force  has  contractor 
provided  primary  pilot  training  at  Williams  Air 
Fora*  Base,  Arizona.  The  Air  Force  also  has  con¬ 
tractor  -support  for  the  C-9  aircraft  and  the  T-43 
trainer.  Hie  Army  lias  a  proposal  for  contractor 
base  support  at  Fort  Fust  is,  Va ,  The  Navy  con¬ 
tracts  tor  software  support  of  Automatic  Trst 
Equipment.  Contractor  suppo* t  is  being  employed 
to  an  ever  increasing  extent  in  the  area  of  direct 
mi  s  s i on  support . 

Regardless  of  the  level  of  support  purchased 
from  the  contractor,  the  customer  must  identify  in 
advance  and  as  accurately  ns  possible,  the  planned 
support  concept  prior  to  initiating  an  RFP, 
Obviously,  there  are  distinct  advantages  to  iden¬ 
tifying  tli  is  during  the  trainer  RFP  phase.  There 
have  been  numerous  examples  where  the  custom' r  has 
changed  his  planned  support  concept  late  in  the 
program.  Tt  is  realized  that  reasons  for  this 
change  may  be  beyond  the  customer's  control. 

Perhaps  a  most  drastic  departure  from  current 
policy  would  be  to  buy  only  a  service  from  the 
contractor.  This  service  would  provide  trained 
aircrews  for  the  customer.  The  contractor  would 
manufacture,  maintain,  operate  and  retain  ownci — 
ship  of  the  device. 

A  progressive  trend  towards  increased  con¬ 
tractor  support  has  distinct  advantages  and  dis¬ 
advantages  for  both  customer  and  contractor. 

Some  advantages  to  the  customer  would  be: 

•  Turn  key  operation, 

•  Decreased  initial  development  tusts , 

•  Decreased  data  costs, 

•  Decreased  manpower- req ui rements . 

•  Decreased  operator  and  maintenance 
training  requirements, 

•  Decreased  supply  support  costs, 

•  Increased  device  availability, 

•  Decreased  O&M  or  RGJ)  funding  costs, 

•  Improved  delivery  schedules,  and 

•  Options  to  buy. 

Contractor  advantages  are: 

•  Design  freedom, 

•  Less  expensive  system, 

•  Less  restrictive  data  requirements, 

•  More  stable  workload, 

•  Flexibility,  and 

•  Responsiveness  to  design  changes. 

As  with  any  beneficial  proposal,  there  are 
inherent  risks  to  both  the  customer  and  the  con- 
tractor.  However,  those  risks  can  be  minimized  by 
including  appropriate  conditions  in  the  contract. 
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*■'!•**  nost  obvious  risks  aro: 

l'o  t.  h  i  •  ous  t  ome  r  : 

•  CotU  rcictor  nonperformance  f  *!«-  f.-ml  t  )  , 

•  bubs  l.  ;tn* Lt  r « I  pe  r  iormaiu e , 

•  Draining  Management  continuity,  and 

•  Solo  source  contracts. 

To  the  Contractor: 

®  Inaccurate  co;;t  *.*s  t  i  nut  os  , 

•  Initial  deve  !  opnu-nt  cost  -  capital 
in vo st ho nt  , 

•  Capital  recovery  factors, 

•  Cost  of  spares,  and 

•  Voudor  support. 

Tu  providing  add  i  t  i  ona  1  assurance  for  success 
with  a  service  program  as  described  above,  the 
customer  may  want  to  consider  competitive  proto¬ 
typing  of  simulator  systems  as  a  means  of  reducing 
consumer  risk.^*4'  because  of  the  contractor's  risk, 
and  the  fact  that  lie  will  be  providing  a  service 
using  his  own  resources,  the  cost  to  the  customer 
will  be  more  realistic  since  unqualified  bidders 
arc  less  likely  to  attempt  to  buy  into  such  a  con¬ 
tract.  This  approach  appears  to  be  only  a  step 
beyond  the  Air  Force  training  program  for  the  KC-10 
aircraft  in  that  title  to  all  training  devices  will 
rest  in  the  Government.  However,  dominion  and 
control  of  all  devices  will  be  retained  by  the 
contractor.  The  major  differences  between  the 
KC-10  approach  and  the  projected  one  is  the  use  of 
an  existing  trainer  as  opposed  to  building  and 
designing  the  simulator  to  a  specific  application. 

A  program  to  develop  a  new  in-flight  refueling 
aircraft  and  associated  support  systems  for  the 
I'nited  States  Air  Force  was  driven  by  DoD  direction 
to  make  maximum  use  of  off-the-shelf  equipment 
it-’Tiis.^)  This  approach  was  designed  to  minimize 
research  and  development  costs  during  the  acquisi¬ 
tion  process.  Any  deviation  from  this  direction 
required  prior  approval.  The  review  process  lead¬ 
ing  to  equipment  selection  was  lengthy,  complicated 
and  enforced  by  government  procurement  directives. 
The  particulars  of  this  process  are  really  unimpor¬ 
tant  for  the  purposes  of  this  paper.  Simply  stated 
from  the  resources  available,  the  Douglas  manu¬ 
factured  DC-JO  aircraft  was  selected  as  the  prime 
equipment  item  for  the  program.  similarities  in 
flight  characteristics  between  the  KC-10  and  the 
DC-  LO  aircraft,  allowed  the  use  of  existing  DC-10 
fiight  crew  training  equipment  and  programs  to 
train  initial  KC-10  flight  crews. 

American  Airlines  had  previously  procured  a 
DC- 1.0  simulator,  developed  an  air  crow  training 
program  and  was  providing  training  to  the  airline 
industry  at  their  Dallas/Fort  Worth  facility. 

Tli rough  Douglas  Aircraft,  an  agreement  was  reached 
with  American  Airlines  to  provide  the  DC-tO  train¬ 
ing  programs  for  Air  Force  flight  crews  on  an 
interim  basis  until  KC-10  training  programs  and 
equipment  could  be  'eve loped.  Subsequently, 
American  Airlines  was  awarded  a  contract  to  procure 


the  necessary  equipment,  and  e  ,s  t  ob  1  i h  ,  qa’iate, 
and  maintain  n  KC-10  training  facility  at.  Bark » dal  e 
Air  Force  Base,  l.A.  5 '  The  equipment,  one*’  procured, 
tested  and  installed,  will  be  cvned  by  the  Air  Force. 
When  the  training  facility  becomes  fully  operation;;!, 
the  contract  with  American  Airlines  will  revert  to 
a  service  contract  with  assorted  options  which  can 
be  exercised  over  a  thirteen  year  period.  The  onlv 
performance  criteria  will  be  to  provide  trained 
KC-10  flight  crews  for  the  Air  Force. 

Research  by  Mr.  Dembroski  and  Dr.  Cans  lor,  as 
reported  in  Nov/Dec  81  Proceedings  of  the  IiUcr- 
servi ce/lndustry  Training  Equipment  Conference, 
indicates  that  the  government  can  expect  long  term 
cost  reductions  in  the  service  portion  of  a  ser¬ 
vice-only  contract.  This  savings  should  result 
from  the  application  of  a  90  percent  learning  curve 
applicable  to  simulators.  Savings  may  be  in  the 
form  of  either  lower  cost  or  increased  per formanco . 
Savings  realized  in  support  costs  are  in  addition 
to  the  obvious  savings  related  to  system  acquisition. 

SUMMARY 

There  are  many  problems  encountered  in  the 
development,  fielding,  and  support  of  a  modern 
flight  simulator.  Problem  areas  exist  in  person¬ 
nel,  training,  spares,  maintenance  and  technology. 
Attempts  have  been  made  to  isolate,  manage,  and 
solve  these  problems  individually  and  collectively. 
Logistics  however,  is  an  interactive  discipline 
encompassing  all  aspects  of  support.  Logistics 
must  play  an  active  role  throughout  they entire 
life  cycle  of  a  system  to  achieve  maximum  cost- 
effectiveness. 

As  this  paper  has  endeavored  to  emphasize, 
the  environment  may  dictate  contractor  support  as 
the  most  logical,  cost-jf fective  support  concept. 

But,  regardless  of  the  support  concept  involved, 
all  requirements  must  bo  clearly  stated  in  the 
early  stages  of  acquisition  and  faithfully  trans¬ 
mitted  through  the  acquisition  documents. 
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ABSTRACT 

The  United  States  has  paid  little  attention  to  nuclear  wanare,  biological  warfare,  and 
chemical  warlare  in  the  last  decade.  Meanwhile,  the  Soviet  Union  initiated  an  expansion  of  its 
chemical  warfare  program  which  has  continued  to  grow  at  a  greater  rate  than  any  aspect  of  their 
military  force.  Soviet  policy  closely  follows  the  words  ot  Marshal  Zhukov  who  said  in  1956, 
"Future  wars  will  not  be  won  with  nuclear  weapons  and  massed  air  power  alone. . .biological  and 
chemical  weapons  will  be  used  to  augment  conventional  and  atomic  warfare."  The  Soviets  and 
their  surrogates  have  used  lethal  agents  i.i  Afghanistan,  Laos  and  Kampuchea.  It  is  clear  the 
Soviets  have  the  capability  and  willingness  to  fight  in  a  biological  and  chemical  environment. 
Soviet  training  programs  integrate  chemical  weapons  systems  with  conventional  and  nuclear 
operations.  v-*Since  conducting  training  in  one  or  any  combination  of  these  three  special 
environments  is  not  possible,  United  States  preparedness  to  fight  on  a  chemically,  biologi¬ 
cally,  or  nuclear  contaminated  battlefield  requires  the  development  of  special  training  equip¬ 
ment,  devices,  and  procedures  which  simulate  these  conditions  as  realistically  as  possible.  To 
accomplish  its  mission,  the  US  Army  must  train  in  peacetime  as  it  will  fight  in  war.  This 
paper  summarizes  the  needs  of  the  US  Army's  requirements  for  training  equipment  and  simulation 
to  train  in  an  NBC  environment.^ 


INTRODUCTION 

Over  the  past  several  years,  the  world  has 
witnessed  the  introduction  of  and  growth  m  the  use 
ot  chemical  and  toxic  weapons  .igaiust  unprotected 
military  ami  civilian  populations  in  several 
wartorn  countries.  Despite  denials  by  Soviet 
officials,  overwhelming  evidence  is  being  accumu¬ 
lated  to  prove  their  complicity  in  and  support  ol 
these  murderous  campaigns.  In  fact,  the  evidence 
has  accumulated  to  the  degree  that  the  United 
States  can  no  longer  ignore  the  threat. 

THREAT 

Despite  its  long  though  sporadic  history, 
chemical  warlare  did  not  become  a  modern  world 
problem  until  World  War  1,  when  a  total  of  114,000 
tons  ot  chemical  agents  were*  employed  producing 
1,297,000  casualties.  The  Soviet  Union  alone 
suffered  some  475,000  casualties,  or  of  the 
total.  It  has  been  suggested  by  some  sources  that 
it  is  these  large  losses  which  psychologically 
drive  the  Soviet  leadership  toward  superiority  in 
this  form  ot  warfare. 

In  1975,  Mr.  .Amos  A.  Jordon,  then  Assistant 
Secretary  of  Defense,  staled  to  the  House  of  Repre¬ 
sentatives  that  "The  Chemical  Weapons  of  the  Soviet 
Union  represent  a  serious  potential  threat."  He 
went  on  to  say,  "We  believe  that  the  USSR  is  better 
prepared  to  operate  offensively  and  defensively  in 
a  chemical  environment  than  any  other  nation  m  the 
world."  General  David  C.  Jones,  then  Chairman  of 
the  Joint  Chiefs  of  Staff,  reiterated  this  position 
in  1981  when  he  concluded  that  "Warsaw  Had  forces 
are  better  equipped,  structured  and  trained  than 
any  other  in  the  world  for  fighting  in  a  chemical 
environment."  There  is  a  great  deal  ot  evidence  to 
suhstant i ate  these  observations. 


There  are  many  uncertainties  inherent  in  any 
estimation  of  the  current  Soviet  offensive  capa¬ 
bilities.  However,  it  is  possible  to  get  a  feeling 
tor  the  scope  of  these  capabilities  by  examining 
some  of  the  following  data. 

The  Soviet  Union  has  a  large  industrial  base 
which  could  he  mobilized  to  produce’  toxic  chemical 
agents.  Sources  indicate  that  at  one  time  some  one 
hundred  chemical  plants  were  m  operation,  ot  which 
half  were  either  producing  or  were  capable  of 
producing  the  latest  agents.  Best  intelligence 
estimates  indicate  that  the  Soviets  have  between 
350,000  and  700,000  tons  of  chemical  agents  ready 
tor  immediate  use.  It  is  further  estimated  that 
Tahun,  a  nerve  agent,  constitutes  50,000  tons  of 
tins  stockpile.  Atlhough  US  intelligence  believes 
that  approximately  one-third  of  the  Soviet  shells, 
rocket  warheads,  and  bombs  stored  in  K  intern  Euro; e 
contain  lethal  chemicals,  other  sources  indicate  as 
much  as  a  50  percent  till.  It  appears  that  the 
Soviet  stockpile  is  7  to  10  times  larger  than  that 
ot  the  United  States,  and  is  sufficient  to  support 
3  to  major  offensives  on  a  wide  front.  The  Threat 
agents  include  the  nerve  agents,  blood  agents, 
blister  agents,  and  toxins. 

Although  the  Soviets  had  difficulty  producing 
toxins,  Tncothecenes  are  .u»w  in  the  Threat  inven¬ 
tory.  The  1.1)  50s  range  from  O.lmg  per  kg  to  lOOOmg 
per  kg,  depending  on  the  particular  toxin  species 
and  method  ot  exposure.  In  addition  to  producing 
casualties,  these  toxins  have  the  advantage  ot 
being  an  effective  terror  weapon.  Disseminated  as 
an  innocuous  powder,  they  cause  blisters  and  vomit¬ 
ing,  as  well  as  severe  bleeding.  Since  they  are 
persistent,  they  also  pose  a  long-term  hazard. 

The  Soviets  have  artillery,  mm  tar,  and  mine¬ 
field  delivery  systems  for  chemical  agents,  as  well 
as  bombs  and  sprays  for  air  delivery.  Chemical 
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rounds  exist  tor  the  122-,  HO- ,  and  152-uii  1 1 imeter 
artillery.  Flu*  BN  21  eons  ists  ot  a  40- tube 
multiple  rocket  launcher  mounted  on  a  l;KAL-  J7r> 
truck  A  oattalion  ran  deliver  720  rounds  ot  the 
chemical-t i l led  shells  in  a  single  ripple  contain¬ 
ing  5  tons  at  agent.  The  I'KOG  and  other  surface- 
to-surface  missiles  allow  the  Soviets  to  extend  the 
reach  ot  their  chemical  weapons. 

Superimposing  the  range  ot  these  weapons  on  a 
stylized  IS  corps  sector,  shows  that  Soviet  mortars 
and  multiple  iocke*t  launchers  tan  attack  division 
targets  from  the  FI.OF  we*  11  into  the  division  rear. 
Targets  include  front-line  troops  and  artiilc*ry 
units  as  we'll  as  some'  logistical  targets.  FROGs 
are  capable  of  attacking  targe'ts  up  to  the*  corps 
rear  boundary,  putt’ug  ivserves  and  supporting 
units,  as  we'll  as  logistical  complexes,  at  risk. 
SCll)  and  tail  leal  air  can  re.uh  transportation 
modes  and  major  logistie.il  support  ce'iiters,  we*il  to 
tlie*  re'.ir  ot  the'  corps. 

An  f*s  t  s  ilia  ?  e'd  80,000  to  Kk),(>0()  ,Sovie*l  chemi¬ 
cal  troop.,  are*  available  to  provide*  support  to 
comh.it  troops  operating  in  a  c  henm  a  1 !  y  coiitami- 
nate'd  environment.  In  the’  ground  toree's,  chemical 
units  are*  organic  down  through  i*e*g iiiie'nt s ,  and 
ehcMiiic.il  pe'rsomie’l  are*  assigned  down  through 
company-s  i  Ze*  units.  Geile-rally,  a  chemical  defense 
brigade'  IS  assign'd  to  a  Sovie't  t  rollt  ,  .:  battalion 
to  o.hIi  army  and  division,  and  a  chemical  company 
to  e'.u  h  lvgi merit .  idiemu.il  servue's  pu>vide*d 
include:  rapid  vrlm  lc,  eloihiug,  |e*rraiu,  and 
runway  de<  out  .uni  nat.  ion ;  warning  ot  attacks,  and 
elumic.il  reconnaissance*  ot  c  out  amin.it  eel  areas. 

lo  augment  their  capacity  tor  t.ist,  highly 
mobile*  wart. ire*,  and  to  prote'et  t  tle'lllSe'  1  ves  I  mm  GW 
agents,  ttle'  Soviet-.  ll.lVe'  ticl.le'd  tilt'  world’s 
largest  and  most  complete  inventory  ot  protective' 
equipment .  Hit'll  prote’et  ive*  masks  piovide  respira¬ 
tory  protection  ..gainst  all  known  eliemical  age'iits. 
Hit’ll  ct>ml> int'd  arms  protceliv.'  suit  tail  be  worn  as 
a  coat,  cape’,  Oi  e  overall.  Although  not  airtight, 
the'  piote'itive  suit  provide:  adequate*  prote'et  ion 
against  most  ctiemii.il  agents.  To  taeilitate*  dett'e  - 
t  1  011  1  dent  i  t  u  at  l  on  and  del  meat  ion  ft  lolls,  a 
variety  ot  alarms  and  automat  u  e  oiitami  nat  i  oil 
marking  means  are  available.  Decontamination 
assets  include  indiviiin.il  kits  and  t  rue  k-niouui  ed 
e*tpi  i  pment ,  nut  ml  mg  tin'  IMS  ,  a  jet  engine- 
particularly  effective  in  rapidly  <h‘i  ontamm.it  mg 
1  arge  veil  i  c  1  e-s  . 

Soviet  p  1  a tlllt* IS  recognize*  that  an  effective 
ibemu.il  capability  depend..  he*.ivily  n  a  c  omprehtni- 
sive  training  program.  Although  precise*  figure's 
au*  not  av.i  i  1  at>  I  e* ,  it  appears  that  IS  percent  of 
the  military  training  given  to  ce.nseiipts  is 
de'Voted  to  NIU  training,  Witll  extensive*  fol  low-oil 
tr.mii ug  being  mte*grate*d  tliroiigtiout  the*  remamde'i 
•  >!  tfie'ir  i  a  r  e*t'  r  s  .  'training  is  d«*signe*«l  to  physi¬ 
cally  and  psyc  liologn  a  1  ty  cond.tion  t.  lit*  lumps  to 
pti  form  their  combat  ass  lgiune-.i!  .  m  full  protective 
gear  . 


Many  Soviet  institutes  and  Industrie's  associ¬ 
ated  with  chemical  and  medual  research  are  also 
iiivolveef  m  chemical  agent  re*search  ainf  develop¬ 
ment  t  KM) ) .  these  programs  are*  government  con¬ 
trol  led,  and  ttie  open  literature  contains  only 
carefully  sanitized  data.  Some*  areas  of  interest 
appear  to  me  fade  efforts  to  synthesize  additional 
G-type  nerve  agents,  as  we*il  as  extremely  toxic 


orgauo-si  J  i  t  runes  .  *1  fie*  isolation,  identification, 

and  synthesis  of  toxi.,s  may  well  le*ai  to  .  new  era 
in  chemical  warfaie*. 

It  is  clear  that  the*  Soviets  can  ..in  cess  t  ul  1  y 
conduct  chcuiiical  warfare.  Itu*  Gig  question  is 
whether  thev  have'  the  will  to  use  these  clie'inicals. 
In  testimony  before  Congress  in  197r>,  J.ieutenant 
General  Howard  d.  Cooksey  answered  this  question 
wilt'll  lie  state'll.  "The  Soviets  are  so  immersed  in 
chemical  weaponry,  tact  its,  doctrine,  equipment  and 
personnel,  and  so  much  ot  their  training  centers 
around  tile'  list'  ot  lethal  agents  tli.it  it  would  he 
odd,  from  a  military  standpoint,  it  they  did  not 
employ  them." 


The  threat  is  nothing  new.  At  the  end  of 
Vorid  War  1,  General  IVrsliing  came  to  the  cone  lu- 
s i on  that  "Whether  or  not  gas  will  be  employed  in 
the  future  is  conjecture,  but  the  t*  t  f  .  is  so 
deadly  to  the*  unprepared  that  one  can  .  afford 

to  ne'gle'Ct  the'  (jiie'.st  i  o;i .  “  The  It.:  ted  ales  has 
neglected  the  thre’.it  over  the*  last  de  e  while'  the* 
Soviets  have  been  and  are*  continuing  ’  prepare  f en¬ 
large  scale*  oiie'iisive*  and  delensive  V  carfare’. 

Hiey  have*  the  agents  and  delivery  men.  and  .ire 
continuing  to  inve*st  in  KM)  efforts  to  ei,  ai.ee 
tfie'ir  capabilities.  Their  military  nmes  are* 
e’Xt  ens  l  Ve’ 1  y  e*qiiippe*d  and  well  trained  tor  opera¬ 
tions  in  a  contaminated  environment.  In  a  nut- 
sliell,  the1  re*  is  a  total  Sovie  t  c  onuni  tnieuit  to  NRG 
war  la  re*. 


Fill:  CHAl.I.KNGh 


Fhe'  IS  Army  re*c  ogn  l  ze*s  tnis  S«.»v  •  e’ t  e  oiunii  tnieuit 
and  re'alizes  that  we  must  tram  to  :ne*e’l  the  line’. it  . 
Since'  we*  do  not  conduit  training  in  an  actual  NRG 
euv  i  ronnie'iit  in  peacetime*  (iinlm**  the*  Sovn’ts),  IS 
response  to  the*  NRG  threat  requi'vs  the'  development 
of  a  s  linn  1  at  e'd-NBC  environment  to  challenge*  our 
forces  and  alien  them  to  tram  as  they  will  have'  to 
tight.  Hie*  IS  Army  Chemical  School  lias  developed  a 
1  r. lining  Device  Acquisition  Strategy  (TI)AS)  which 
MiiKiu.i r i i’es  the*  nei'essary  concept  to  support 
training  m  a  s  min lated-NHC  environment.  Ibis 
document  will  serve'  as  the  principal  instrument  tor 
ifire'iting  the'  Army  efforts  to  coin  ept  1.1 1  i /e , 
de'Ve*  lop,  and  ae  quire  training  equipment,  simulants, 
and  *  upper  t  pac  kages/proce*ifijre*s  tor  tins  training, 
liii'  IDAS  supports  the*  TRADGG  goals  of  a-,  lneving 
substitution,  simulation,  and  miniaturization  where* 
t  e’.is  i  b I e/app I  1  cab  1  e* .  1‘hus  far  we  1  . . V e '  ide'llt  I  t  le'el 
22  proposed  device's  oi  systems  for  the'  IDAS. 


SI  M.MAKY 

Fhe*  systems  and  devices  (see'  annex)  discussed 
m  tins  paper  ofler  a  challenge  to  all  who  are' 
interested  in  establishing  a  realistic  and  Viable* 
NRG  training  environment.  We’ve  identified  the 
threat,  defined  our  requirements,  and  this  conven¬ 
tion  has  provided  t fit*  forum  to  make*  our  needs  known 
to  von,  tin*  uinoVat  oi  s .  Accept  jud  nie*et  this  chal¬ 
lenge  and  our  fighting  forte's  will  immeasurably 
improve  the*u  e  apabi  1 1 1  ie*s  in  an  NBC  environment. 


ANNEX 

1.  Niu  lea r  Weapons  Effects  Simulator 

a.  Tramiiig/Simulat  ion  Requirement.  Simulate 
fl.sh,  bang,  neutron  and  gamma  radiation,  mushroom 
c  loi. ',  elec  tromagnetic  pulse  (EMR),  and  transient 
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radiation  electronics  effect  (TREE);  exercise  troop 
reaction  to  nuclear  burst;  exercise  NBC  Warning  and 
Reporting  System  (NBCWRS);  exercise  C2  procedures. 

b.  For  Use  At:  Armywide,  National  Training 
Center,  US  Army  Chemical  School. 

c.  Character i st i cs  of  Desired  Device/Fac i 1 1 ty . 

(1)  Flash  must  be  distinctive,  observable 
under  combat  conditions,  with  a  range  of  5  to  15 
ki lometers . 

(21  Bang  must  be  distinct,  distinguishable 
Iron)  battle  sounds,  with  a  range  ot  5  kilometers. 

(11  Time  between  tlash  and  bang  must 
parallel  that  of  a  nuclear  weapon  tor  use  in  the 
NBC  reporting  system. 

(4)  Simulate  real  time  and  realistic 
generation  ol  prompt  neutron  and  gamma  radiation. 

'  (r>)  Produce  a  mushroom  cloud  with  meas¬ 

urements  and  duration  ('fleet  comparable  to  the 
measurements  ol  ’ -KT,  S-KT,  and  20-KT  weapons. 

lb)  Simulate  electromagnetic  pulse  (KNP) 
and  transient  radiation  electronics  el  ted  (TREE). 

d.  Status. 

(1)  Fielded:  The  M142  Nuclear  Burst 
Simulator  is  inadequate.  Troops  do  not  recognize 
it  as  a  unclear  burst,  part i cularly  m  vehicles. 

It  is  not  distinctive  under  combat  conditions  aid 
does  not  simulate  vaii.it  ion  el  feels.  Exercise  ol 
NBCWRS  is  marginal. 

(2)  In  development:  Preliminary  study  by 
Sc  i  eiu  e  App  lit.itimi,  1  nc  . 

(1)  Proposed  device  system:  Ninle.n 
we  ipous  elicits  simulator  (NWKS). 

2.  lot  a  I  Dose  l)ose-i\at  Simulator 

a.  Training  Siiniil.it  i  -:i  Requirement:  Simulate 
dose  rates  and  total  dose  from  i  ndiiced/ I  a  1 1 1  ul 
radiation,  as  well  .is  initial  dose-rate  readings. 

b.  F»r  l  se  At:  Arinywide,  Nat  i  <-n.il  rr.iiniiig 
Center,  IS  Army  Cliemi ca  1  Silnu-1. 

i  .  Clia  i  ad  e i  i  st  l  cs  •  t  Desired  l)e\  i  ce  1  ii  i  I  1 1  y 

(1)  Simulate  real  time  ami  a.ilis  ta 
leadings  <-n  rad i ai meters  in-1  -losimeteis. 

(2)  H-tal  <l-»se  ami  <lose  i.ite  must  be 
measurable  over  time  and  loi.it  loll. 

<1.  Status. 

(1)  Fielded:  I  In-  AN  IDM-litV)  truinei 
only  interlaces  with  th<‘  1M-I’4  i  a-1 1  .miiet  er  and 
simulates  lose  late  •  >n  I  y  -  l«>tal  dose  is  in-t 
me. i mi i  ab  1  e  . 

(2)  In  development:  None. 

(>)  Pi  oposed  ilevi  *«• --'system:  l<»tal  dose 

dose-rale  simulator  (1  DDKS  I. 


3 .  Radiation  Automatic  Casualty  Assessment  System 

a.  Training/Simulation  Requirement:  Assess¬ 
ment  arid  assignment  of  initial  and  delayed  radia¬ 
tion  casualties. 

b.  For  Use  At:  Armywide,  National  Training 
Center,  US  Army  Chemical  School. 

c.  Characterist its  of  Desired  Device/Facility: 

(1)  The  system  should  be  near  real  time  and 
provide  total  simulated  casualties  and  radiation 
doses . 

(2)  The  system  should  be  realistic  and 
indicate  to  personnel  that  they  are  casualties. 

(3)  For  delayed  casualties,  the  system  must 
be  realistic  and  provide  total  simulated  incapaci¬ 
tation  and  total  dose  for  each  individual. 

(4)  The  system  should  provide  assessment  ol 
degradation  on  performance  as  a  result  ol  radiation 
exposure  and  a  time  tag  lor  incipient  casualties  to 
be  i ncapac i t ated  at  a  later  time. 

d.  Status: 

( 1  )  F i e 1 ded :  None . 

(2)  In  development:  None. 

(3)  Proposed  Dev  a  c/System:  Radiation 
Automatic  Casualty  Assessment  System  (RACAS). 

4.  Biological  Agent  Simulant 

a.  Pi  a i n i ng  S  nau I  a 1 1 on  Requirement.  Simulate 
the  effects  of  hi  'logical  agents  to  include  toxins 
use-l  i<y  llireat  lories;  icaition  to  biological 
agent;  and  equipment  ami  pc  i  soilin'  1  <iei  oiit.inu  ii.it  ion 
t  i  a  iiimg. 

b.  For  Use  At:  Arinywide,  National  I  ruining 
Center,  IS  Army  Cl.-.-ma  il  school  . 

i.  Characterist  a  s  of  Desired  Dev  a  e/ f-  .u  i  1 1 1  y  . 

ill  Hie  s  i  mu  I  .nit  agent  or  family  of 
simulate  agents  must  realist  aally  simulate  the 
giiysa.il  propel  ties  of  llireat  agents. 

12)  It  must  he  suitable  f-»i  equ  I  pnie'i.  ami 
personnel  dec  oiilamni.it  am  ti  lining,  using  :  t  tmlua! 
pi  *k  (.*  <  1 1 1 1  es  with  a  <lei  eill  ami  lilt  l  oil  .simulant  . 

1  ’)  Hie  simulant  must  iont  am  a  ti.m-i 
i.ip.ible  o|  eas\  removal  for  use  in  evalu.ita-n  «  t 
dt'i  ont.iilllM.it  1>'|I  tl. lining  groi  edui  es  . 

t  •<  )  lhe  simulant  must  be  i.ip.ible  ot 
qiie-n  fi i  tig  itsel  t  and  he,  oiiie  iioii.ntae  4  t<>  flours 
after  app  1  a  a  t  i  on .  s«>  the  training  a  tea  nay  be  used 
aga-n  without  i  lit  e  i  I  er 'em  e  . 

•i.  Status. 

(  l  t  la  1-le-t:  Nolle. 

(2)  in  development  ;  None. 

(tl  Proposed  device  system:  Ba-loga.it 
agent  simulant  i  K-\S  )  . 
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5.  Biological  Agent  Casualty  Assessment  System 

a.  Tranung/Simulation  Requirement.  Simulate 
the  assessment  of  biological  agent  casualties. 

b.  For  Use  At:  National  Training  Center. 

c.  Essential  Characteristics. 

(1)  The  system  should  work  in  conjunction 
with  existing  and  future  biological  agent  alarms. 

(2)  The  system  should  work  in  conjunction 
with  individual  masks  and  collective  protection 
systems . 

(3)  The  system  should  assess  and  assign 
biological  casualties. 

(4)  The  system  should  indicate  to  personnel 
that  they  are  casualties. 

(5)  The  system  should  assign  and  assess 
incapacitated  personnel. 

d.  Status. 

(1)  Fielded.  None. 

(2)  In  development:  The  simulated  area 
weapons  effects  <SAWE)  project  currently  being 
performed  by  Jet  Propulsion  Laboratory  under  con¬ 
tract  to  PM  Trade  to  determine  the  best  technical 
approach  (BTA)  to  simulate  the  effects  of  area 
weapons  during  lorce-on-iorce  training  exercises. 

(3)  Proposed  devi ce/ fac i 1 i tv :  Biological 
agent  casualty  assessment  system (BACAS ) . 

b.  Biological  Detection  and  Alarm  Training 

Simulator 

a.  Trairnng/Simulat  ion  Requirement.  Activate' 
biological  detection  and  alarm  systems  in  a  train¬ 
ing  environment;  properly  employ  biological  alarm; 
react  to  biological  attack. 

b.  For  Use  At:  Armywide,  National  Training 
Center,  US  Army  Chemical  School. 

c.  Characteristics  of  Desired  Dev  ice/ Fac i 1 ity 

(1)  Must  have  a  built-in  training  mode  in 
addition  to  the  operational  mode. 

(2)  The  training  mode  must  not  defeat  the 
operational  mode. 

(3)  The  training  mode  must  he  built  in  or 
allowances  must  be  made  for  it  in  ail  operational 
systems  under  development. 

(4)  Must  be  capable  of  being  activated 
remotely . 

d.  Status: 

(1)  Fielded:  None. 

(2)  In  development:  None. 

(3)  Proposed  device/ system:  Biological 
detection  and  alarm  training  simulator  (BOATS). 


7 .  Biological  Agent  Decontamination  Simulant 

a.  Training/Simulat’on  Requirement.  Decon¬ 
taminate  biological  agent  simulators. 

b.  For  Use  At:  Armywide,  National  Training 
Center,  US  Army  Chemical  School. 

c.  Characteristics  of  Desired  Device/Facility: 

(1)  The  decontamination  simulant  mast 
decontaminate  the  biological  agent  simulant. 

(2)  The  simulant  must  be  able  to  be  used  *n 
all  approved  and  developing  decontamination  systems 
and/or  their  associated  training  devices. 

(3)  The  procedures  used  to  decontaminate 
must  duplicate  the  procedures  used  to  decontaminate 
Threat  agent . 

d.  Status: 

(1)  Fielded:  None. 

(2)  In  development:  None. 

(3)  Proposed  device/system:  Biological 
agent  decontamination  simulant  (BADS). 

8.  Chemical  and  Biological  Agent  Delivery  Ssstm 

a.  Training/Simulatiou  Requirement:  Simulate 
delivery  systems  for  Threat  chemical  and  biological 
agents . 

b.  For  Use  At:  Armywide,  National  Training 
Center,  US  Army  Chemical  School. 

c.  Characteristics  ol  Desired  Devi ce/Faci 1 l tv: 

(1)  The  system  must  provide  the  same  cues 
the  Threat  system  provides. 

(2)  The  system  must  duplicate'  the  disper¬ 
sion  patterns  and  area  coverage  of  the  Threat 
system. 

(3)  The  system  should  be  reusable  and 
easily  filled. 

(4)  The  system  should  make  use  of  a  per¬ 
sistent  chcm.cal  agent  simulant  and  a  biological 
agent  simulant. 

»f.  Status: 

(1)  Fielded: 

(a)  M9  SPAL  has  a  limited  ability  to 
simulate  artillery-delivered  chemical  agents.  Arra 
coverage  is  insufficient. 

(b)  Aero  14B  aerial  spray  tank  is  in 
the  inventory.  It  can  be  used  to  disseminate 
liquid  chemical  agents.  !t  is  not  compatible  with 
mult l service  tactical  aircraft. 

(c)  The  Mb  not  control  agent  dispenser 
is  limited  in  its  ability  to  deliver  chemical  agent 
simulants . 

(2)  In  development :  A  training  system  tor 
chemical  defense  (Phase  11),  using  the  XM2t>? 
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launcher  and  XM11  SPAL  simulates  artillery- 
delivered  chemical  agents. 

Restrictions  on  emplacement  and  use  for  safety 
reasons  limit  the  effectiveness  and  realism  of  the 
system,  making  the  training  effects  questionable. 

(3)  Proposed  device/system:  Chemical  and 
biological  agent  delivery  system  (CBADS). 

9 .  Nonpersistent  Chemical  Agent  S imu lant 

a.  Training/Simulation  Requirement:  Simulate 
nonpersistent  chemical  agerit/agents  used  by  Threat 
forces;  react  to  chemical  attack. 

b.  For  Ure  At:  Armyvidr,  National  Training 
Center,  US  Army  Chemical  School. 

c.  Characteristics  of  Desired  Device/Facility: 

(1)  The  simulant  agent  or  family  of  simu- 
•ant  agents  must  realistically  simulate  the 
physical  properties  of  Threat  agents. 

(2)  The  simulant  must  be  capable  of  quench¬ 
ing  itself  and  becoming  nonactive  4  to  24  hours 
after  application,  so  that  the  training  area  may  be 
used  again  without  interference. 

d.  Status: 

(1)  Fielded:  N-buty l -mercaptan  (HUSH)  does 
not  simulate  the  physical  properties  of  Threat 
agent /agent s .  CS  gives  a  misleading  cue. 

(2)  In  development:  None. 

(.})  Proposed  dev i  ce/ system:  Nonpersistent 
chemical  agent  simulant  (NCAS). 

10.  Persistent  Chemical  Agent  Simulant 

a.  framing  Simulation  Requirement:  Simulate 
persistent  ch<*miiul  agent/agent s  used  by  Threat 
tones;  reai  t  to  chemical  agent;  suitable  for 
equipment  am.  personnel  del  out  ami nat ion  training. 

b.  For  Use  At:  Armywide,  National  framing 
Center,  IS  Army  Chemiul  Sihool. 

c .  Charm  t»*r  i  st  1 1  s  of  Desired  Dev  i  ce/'f.ic  i  I  1  tv : 

11)  Hie  simulant  agent  or  family  ot  simu¬ 
lant  agents  must  rea  1 1 si  1 1  a  1  1 V  simulate  tile  physi¬ 
cal  p rope i l i es  ot  threat  agents. 

1 2 )  It  must  he  suitable  for  equipment  and 
personnel  dei  oiilamin.it  ion  training  using  standard 
proi  edore:.  with  a  det  out  ami  nat  I  <>n  simulant. 

if)  The  simulant  must  loiitam  a  t  rater 
tip.ible  ut  easy  i  emova  1  t  ,u  use  in  evaluation  of 
dei  ontami  nat  ion  training  pi  <u  edinrs  . 

(4)  the  simulant  must  hr  tapahir  of  quench* 
mg  itself  and  bni-ming  uoiiailive  4  to  2-*  hours 
after  application  so  that  the  training  area  m.iv  be 
used  again  without  I  liter  ferrtn  e  . 

( f> )  Th**  si  mu! -tit  used  l<  simulate  mustard 
ga  s  should  lontain  III  addition-  to  the  quill  lies 
listed  previously,  a  time  lapse  development  capa¬ 


bility  to  sirulate  the  delayed-casualty  effects  of 
mustard  agents. 

d.  Status: 

(1)  Fielded:  Polyethylene  glycol  (PEG)  200 
does  not  simulate  the  physical  properties  of  Threat 
agents.  It  is  not  suitable  for  evaluation  of 
decontamination  training  procedures  and  is  not 
capable  of  time  lapse  development. 

(2)  in  development:  Training  system  for 
chemical  defense  (Phase  II)  delivers  PEG  200  using 
the  XM267  launcher  and  the  XM11  simulator  projec¬ 
tile  airburst  liquid  (SPAL).  Restrictions  in 
emplacement  and  use  for  safety  reasons  limit  the 
effectiveness  and  realism  of  the  system,  making  the 
training  effects  questionable. 

(3)  Proposed  device/system:  Persistent 
chemical  agent  simulant  (PCAS). 

11.  1R  Smoke  Simulator 

a.  Trim  mg/ Simulation  Requirement:  Provide  a 
training  device  that  will  demons',  rate  the  effects 
of  infrared  ( !R)-defeat  ing  suiukf. 

b.  For  Use  At:  Armywide,  National  Training 
Center,  US  Army  Chemical  School. 

i  .  Char  icter i st i cs  of  Desired  Device/Kaoi 1 ity: 

(1/  1R  sighting  devices  that  simulate 
viewing  into  IR-defeat mg  smoke. 

(2)  When  ,.sed  in  conjunct  ion  with  visual 
range  smoke,  .  *.  will  simulate  t  lie  fill!  range  of 
(IR-vi  snail  sutoke  tupahility. 

d.  Status: 

ll)  Fi e 1 ded :  None . 

(2)  In  development:  None. 

(f)  Proposed  devu  e/ system:  IK  smoke 
s imu 1  at  or  ( i RS3  ) . 

12.  Clieniu.il  Agent  Casualty  Assessment  System 

a.  t'raiiimg/Sinuilat  ion  Requirement.  Simulate 
the  assessment  of  ihetnii.il  agent  iasii.il  ties. 

h.  for  I  sr  At:  National  (Taming  Centei 

i.  E  .sent  ia  1  Ch.ir.ii  tri  i  st  1 1  s  . 

ill  l  he  system  should  work  m  i  on  jum  l  i<m 
with  existing  and  tiituie  i  lintm.il  agent  alaims. 

(2)  The  system  should  work  in  i «  n  |iiin  t  i«-n 
with  individual  masks  and  collective  proteition 
systems . 

(U  1  he  system  she  .Id  assess  and  assign 
iliemii.il  casualties. 

(4)  Ihr  system  should  i  initiate  to-  pei  si.nnr  1 
that  they  are  i asua 1 1  les . 

( N )  file  system  should  assign  and  assess 
llii  ap.it  1  l  aled  pei  soiinel  . 
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<1.  Status. 


(1)  Fielded:  None. 


(1)  Fielded:  None. 

(2)  In  development:  The  simulated  urea 
weapons  et facts  (SAWE)  project  currently  being 
performed  by  Jet  Propulsion  Laboratory  under  con¬ 
tract  to  PM  Trade  to  del  ermine  the  best  technical 
approach  (BTA)  to  simulate  the  effects  of  area 
weapons  dining  t orce-on-torce  training  exercises. 

(.1)  Proposed  device/facility:  Chemical 
agent  casualty  assessment  system  (CACAS). 

I  t .  Chemical  Detection  and  Alarm  Training 

S i mu  1  at  or 

a.  Training/Si*  uniat  i  or.  Retjui  rement  .  Activa¬ 
tion  of  i hemirn I  detection  and  alarm  systems  in  a 
training  environment;  pripei  employment  of  chemi¬ 
cal  alarm;  and  reaction  to  chemical  all.uk. 

b.  lor  Lse  At:  Armywide,  National  framing 
Center,  US  Army  Chemical  School. 

c.  Character i st  it  s  of  Desired  Device/Eac  i  I  ity : 

(1)  Must  have  a  bn  ill- m  training  mode  in 
addition  to  the  operational  mode. 

(2  1  file  training  mode  must  not  defeat  t  lit' 
operational  mode. 

({)  fhe  training  mode  must  he  built  m  or 
allowances  must  In*  made  for  it  m  all  operational 
systems  antler  development. 

(o)  Must  be  capable  of  remote  at tivaliou. 

d.  Status: 

it)  1  i  e  1  .led :  None . 

I J  )  hi  development:  Hie  .VMS  1  simulator  is 
ette.tive  with  certain  limitations.  file  system 
w  1 1  ft  the  training  device  oil  it  does  not  replicate 
tin1  actual  system. 

■  >)  Proposed  dev i c e/svsl em :  Chemical 
itt'tec  t  ion  and  alarm  training  simulator  (CPAIS). 

1  •« .  (Mitumcal  Agent  Dec  out  ami  n. it  ion  Simulant 

a.  l'i  a  i  n  i  ng<  S  iinu  1  at  i  on  Ke«ju  i  i  t*m«*n  t  .  Decon¬ 
taminate  persistent  chemical  agent  simulants. 

!>.  foi  lse  At:  Armywide,  National  IT. lining 
C.ntei,  IS  Airny  Chemical  School. 

c.  Cliar.u  terist  us  c>  t  Desued  Dev  n  e/ ku  i  J  i  ty  : 

(I)  flu'  decontamination  simulant  must 
dec  c»nt  am  male  the  chen.'iai  agent  simulant. 

(J)  The  simulant  must  be  ade  to  be  used  in 
all  approved  and  developing  dec  out  ami  n.il  l  oil  systems 
and  or  then  associated  training  devices. 

1  f)  The  proposed  simulation  procedures  must 
duplicate  the  actual  piocedures  used  to  decontami¬ 
nate  the*  threat  agent  . 

d.  Status: 


(2)  In  development:  None'. 

(3)  Proposed  device /system:  Chemical  agent 
decontamination  simulant  (CADS). 

iS.  Trainer  Jet  Kxhanst  Dec  out  ami nat i on  System 

a.  Trainiug/Simulat ion  Recpii rement .  Simulate 
the  operational  and  maintenance  characteristics  of 
the  jet  exhaust  decontamination  system  (Jhl)S). 

h.  For  Use'  At:  US  Army  Chemical  School. 

c .  Characterist i cs : 

(1)  The  training  di'vice  mast  provide  the 
operator  the  opportunity  to  become  proficient  in 
the  operation  of  the  JEDS. 

(2)  The  device  must  be  able  to  duplicate 
the  preoperational,  operational,  and  postoper.it  ion- 
.il  checks  and  procedures. 

(1)  The  device  must  lie  a  reasonable 
facsimile  of  the  decontamination  operator’s  c  ih  of 
the  JEDS. 

(a)  l .Tumi  >pe*rating,  the  device  must  operate 
and  sound  I i ke  an  actual  JEDS.  Additionally,  the 
operator  must  be  able  to  see  a  reasonable  facsimile 
of  the  equipment  being  decontaminated. 

(S)  The  l  .*ain mg  devic  e  must  provide* 
maintenance  personnel  i  lie  capability  to  trouble¬ 
shoot  and  repair  the  JEDS. 

d.  Status: 

til  Fielded:  None. 

(2)  In  development:  None. 

(i)  Proposed  device/systrm:  Trainer  let 
exhaust  decoiitann  uat  »on  system  (TJFI)S). 

lt>.  Chemical  Muipliou  IT  a  i  mug  Devices 

a.  T ra i in ng/ S imu 1  at i on  Requirement.  Deve'op 
ti  lining  devices  for  US  Army  munitions. 

b  For  Use  At:  Armywide. 

c.  Charact  ei  i  st  i  c  s  of  Desired  Dev  i  c  e/Fac  i  1  1 1  y  : 

(1)  Hie  l  i.i  i  II  i  up./  s  mm  1  at  i  oil  device  must  be 
identical  in  physical  appearance*  to  the  actual 
system  it  is  intended  to  simulate. 

>2)  The  training  device  mils!  be  similar  in 
operation  to  the*  actual  system. 

t.U  The*  device  must  provide  realistic 
training  for  art  1 1  lei  y,  IT  H) ,  and  ammunition*' 
handling  pet  sonnet. 

d.  Status. 

( 1  )  Kiel  ded  :  None* . 

(2)  In  development:  None. 
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(3)  Proposed  devic -'/system: 

(a)  Training  binary  chemical  warhead: 
Multiple  launcher  rocket  system  (NI.RS). 

(b)  Training  projectile  1 5  a -mm  GB-2. 

(c)  Training  chemical  warhead :  Corps 

support  weapon  system  (CSWS). 

(d|  Training  projectile  In  nary 

intermediate'  vohtility  agents  (IVAf, 

(t*)  Training  projectile  8-itich  VX-2 
( I VA ) . 

17.  NBC  War  Games  and  Simulation  Center 

a.  Tra i n iug/S imu I  at  1  on  Requirement.  To 
provide'  students  with  realisfie  simulation 
(physical  and  mental  sensat  i oas )  ol  NBC  el  teds  and 
combat  in  ail  NBC  environment. 

I).  For  Use  At:  US  Army  Cliemic.il  School 

c.  Characteristics  ot  Desired  Dev i ce/Fac i i i ty . 

(I)  Floor  space  measuring  33,000  square' 

leet  . 

(a)  Three*  large,  3 ,000-square-loot 
battle  simulation  Towns. 

|ll)  Four  smaller,  .1 ,000 -square -1  not 
battle  simulation  rooms. 

(c  •  Storage  space  measuring  10,000 
square  feet  for  props,  mock-ups-*  chemicals,  paper 
goods,  elect  r<>n ic  hardware',  and  computer  and  audio¬ 
visual  sot  t  ware'. 

(d)  Up  - 1  o  -da  t  e*  military  e  omnium  cat  i  on 

equ l pment . 

(3)  Access  doors  into  larger  studios  and 
storage  a  teas  adequate*  tor  entry  of  large*  vehicles. 

(  J ;  hi-house  communication,  e  lose'd-c  l  reni  t 
TV  (large*  sc  re*e*n ) ,  computers  with  display  terminals 
and  printers,  and  sound  system. 

(4)  Maneuver  and  parking  space  lor  display/ 
mot.  k  vehicles 

(r>)  Soundproofing  throughout  building. 

(t>)  Computer  link  with  o'.he-i  stations. 

(7)  High  output  sound  system.. 

(8)  Sensor-round  battle*  theater. 


(2)  Proposed  Facility:  A  gam i ug / s imu I  a t l on 
tacility  supported  by  a  core  of  in-house  game  play¬ 
ers,  instructors,  and  production  and  technical 
personnel .  Tliis  section  must  be  composed  of 
individuals  with  a  wel  l-rOgtided  knowledge  of  NBC 
doctrine,  tactics,  and  methods  of  instinct  ion. 
KvetH.ua!  ly,  tins  section  will  have-  to  he  able  to 
export  gar.ies/s  imul.it  ion  to  other  NBC  schools. 

18.  NBC  Kvalu.it  ion  Training  Facility 

a.  Tra i n i ug/ S imu I  at i on  Requirement:  Exercise/ 
evaluate  NBC  units  in  uccompj i sh i ng  NBC-related 

m i ss i ous . 

b.  For  Use  At:  US  Army  Chemical  School. 

c.  Character i si i cs  of  Desired  Dev i ce/Fac : 1 i t y . 

(1)  An  exert  i se/eva 1 uat ion  section  to  plan 
for  and  exercise/evalnate  designated  corps  and 
divisional  NBC  units  in  NBC  support- re  I ated 
missions. 

( 2 )  Equipment  and  personnel  decontamination 
training  to  he  conducted  in  t lie  fie* id. 

(1)  hive-agent  decoiilaniinat  ion  training  to 
he  conducted  at  the  t  hemie.il  decontami uat  i on  train¬ 
ing  t  ae i 1 i t  y . 

(4)  Nuclear  ret  onua  i  ss.iucr  to  he  conducted 
in  a  simulated  nuclear-burst  training  area. 

(r>)  A  chemical  reconnaissance  training  area 
to  be  established,  using  chemical  simulants  t< 
dc't  ec  t  /  l  dent  i  *  v  chemical  agents. 

d.  Status. 

(1)  Operational:  One  decontamination 
training  site  exists  l<>r  instruction  .it  the  IS  A  tiny 
Chemical  School.  Hi  is  is  insufficient  lor  unit 
needs* . 

I/)  Under  deve I opment :  A  live-agent  t  tam¬ 
ing  tac  llily  has  been  designed  and  budgeted  lor. 

(<)  Proposed  tacility  range:  A  nuclear  and 
rec  c'lin  1 1  ssance  training  .  *ea  is  proposed.  Hus 
range  is  to  be*  integrated  w  1 1  h  requirements  associ¬ 
ated  with  ,‘he  present  team  facility  and  live-agent 
t  ra  mi  ug  f  jc  i  I  i  t  y . 

Id.  Nucl  ear  Arc  i  deul  /  i  uc  l  dei.l  Control  Chemical 

Accident  Incident  Control  I’l.nniug  Site 

a  t  ranting  Recjn i  remeut  :  Piovide  a  ti. lining 
area  loi  unclear  and  chemical  accident  incident 
c  on  1 1  o  1  traiiniig.  (Wfien  the-  IS  Aimy  Chemical 
School  becomes  the  1)01)  Chemical  School.) 


(d)  Kec  1 1  c  u  I  a  1 1  ug ,  .ntivated-charcoal- 
tiltered  a  i  i -c  oinh  i  lou.ng  system  lor  at  least  one 
battle  simulation  room.  Allows  use  of  chemical 
s  imu  1  aril  s  . 

(Kit  Holographic  hurduair  for  l-D  stmiua- 

t  i  on . 

d .  Stains: 

(I)  Fielded.  None.  (Similar  type  facilities 
exist  at  Forts  Benniiig,  Rucker,  and  Nnox  ) 


h.  for  Use  At:  IS  Army  Chemical  School  by 
.students  and  area/ PARCUM  NAfC/CAlc  te,.ms. 

c.  Cliarac  tensl  ic  s  of  Desired  Device  Facility: 

ll)  Accident  sites  approximately  «  to  t> 
squaie  kilometers  each. 

(2)  NAIC  site:  Several  * .fferent  niuleir 
weapons  tiaimng  device*,  available  to  be  placed  on 
the  site.  "Seeding"  w  th  Alpha  emitters  to  be 
v  ons ide red . 
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(3)  CA1C  &ite:  Accident  scene  reflects 
environmental  release  of  tocic  chemicals.  Effects 
of  chemicals  to  he  simulated. 

(4J  Several  scenarios  developed  to  test  the 
NAIC/CAIC  team  procedures  for  rendering  safe, 
securing,  cleaning  up,  and  controlling  an  accident 
site. 

d.  Status: 

(1)  Fielded:  Kirtland  AFB,  New  Mexico,  has 
a  good  NA1C  site.  Should  the  DOD  Chemical  School 
concept  be  approved,  transfer  of  this  mission  from 
Kirtland  AFB  to  Fort  McClellan  would  he  considered. 

(2)  Under  development:  None. 

(.3)  Proposed  Device/System:  NAIC/CA1C 
training  site. 

20.  N'BC_  Compute r-Ass_is ted  Training  Module 

a.  T.  uni  ng/Si  mulct  ion  Requirement.  To 
develop  an  NBC  computer-assisted  training  (NBC  CAT) 
module,  similar  in  concept  to  the  hr igade/battal ion 
level  administrat ive  or  logistics  module  for  battle 
simulation  and  war  games  or  to  First  Battle. 

Purpose  is  to  better  train  all  individuals  required 
to  know  NBC  command  and  cnn'rol  techniques . 

h.  Character i st i cs  ol  Desired  Device/Faci 1 ity. 

(1)  Flexible  in  design  s<>  that  different 
levels  of  c oiranand-- corps ,  divisions,  brigade, 
battalion,  and  company-can  practice  NBC  skills. 

(2)  Flexible  in  design  so  that  different 
branches  can  relate  to  chemical  module  m  term.,  ot 
manpower  and  types  ot  equipment. 

c.  Current  Field  System.  Numerous  war  games 
are  available  now.  Three  are  noted  above,  but 
there  are  others  that  could  be  used. 

d.  Concept  ol  Development.  NBC  CAT  module 
content  and  concept  should  be  developed  and 
coordinated  so  expert  consultation  and  agreement  is 
realized  between  the  ISACMLS  game  maker  and  i ranch 
use  i  s . 

21.  Multimedia  SBC  Threat  Module 

a.  Ir. 1 1  in  ug/S  i  mu  lit  i  on/ Support  Uequi  remen  t  . 
Provide  a  classified  NBC  threat  brief  and  an 
uni  ! ass i  1 1 ed  tlirejt  brief  to  Chemical  School 
students,  visiting  VIPs,  mobile  training  teams,  and 
tor  expo r t-.d‘ ie  use. 

h  lot  Use  At:  IS  Army  Chemical  Softool,  area 
NBC  MiK'nis,  MU  sections,  servne  schools. 

».  Ch.ti.it.  ter  isl  tes  uj  Desired  {)ev  i  <  e,  fat  i  1 1 1  y : 

til  Hut  h  the  unclassified  a|ld  it  ass  It  led 
hi  ie{  i  rig  would  consist  <>{  a  NO-mmute  multimedia, 
audiuvise.!  presentation  that  discusses  the  enemy's 
of  tonsil  i ltd  defensive  capabilities  .»»<*  presents 
lineal  doctrine  oil  the  .tit-land  hattletlrlf. 

(2  1  Both  briefings  should  exploit  the 
en’.ii*  spectrum  of  audiovisual  rlfeits  t<-  prodme  a 
quality  product  fr.g..  jit-iar>d  2000  hinting). 


d.  Status: 

(1)  Fielded: 

(a)  The  US  Army  Chemical  School  has  a 
relatively  primitive  classified  and  unclassified 
threat  briefing. 

(b)  Intel l ipence  and  Threat  Analysis 
Center,  1NSC0M,  provides  a  classified  briefing  to 
COAC  personnel. 

(2)  In  development:  None. 

(3)  Proposed  D. vice/System: 

(a)  NBC  threat  briefing  (classified). 

(b)  NBC  threat  briefing  fund  ass i f red ) 
22.  Scale  Model  NBC  Equipment 

a.  Training/Simulation  Requirement:  Compati¬ 
ble  scale  models  of  power-driven  decontaminating 
apparatus  and  smoke  generators  are  required  to  de¬ 
pict  NBC  units  during  terrain-board  and  sand-table 
exerc i ses . 

h.  For  Use  At:  US  Army  Chemical  School, 
service  schools,  units  (managed  by  TASt? ) . 

c.  Character isl ics  of  Desired  Device/ I.k  t  I  .  ty 

(1)  Models  should  be  made  ol  high- impact 
plastic  and  in  compatible  scales  ot  available 
tact  i  c  a  I  V‘*h  i  c  i  es  . 

(2)  Moving  pacts  are  not  required. 

(I.  Status: 

(  1  )  Kiel  di'd  :  None  . 

(2.‘  Under  Development:  None. 

(1)  Proposed  Device/System:  Scale  model 
MU  equipment. 
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'"'Some  of  the  findings  of  AGARD  FMP  WG10  are  discussed.  Image  detail  and  resolution 
are  two  characteristics  of  visual  simulation  systems  which  seem  to  be  significant  in  deter¬ 
mining  pilot  performance  in  a  simulator.  MTFA  has  been  found  to  be  a  reliable  metric  for 
predicting  target  acquisition  performance,  however,  present  measurement  techniques  are  not 
suitable  for  moving  imagery.  Reliable  techniques  for  the  measurement  of  both  image  detail 
and  resolution  need  to  be  developed.  Apparent  motion  as  opposed  to  real  motion  is  also 
discussed.  Neither  the  effect  of  visual  simulation  characteristics  on  apparent  motion  nor 
the  effect  of  apparent  motion  on  pilot  performance  are  well  understood  and  require  further 
research 
/i 

\  INTRODUCTION 


Working  group  10  was  established  by  the 
flight  Mechanic's  Panel  (FMP)  of  the  Advisory 
Group  for  Aerospace  Research  and  Development 
(AGARD)  in  March  1979  to  consider  and  report  the 
characteristics  of  flight  simulator  visual  sys¬ 
tems.  ( 1 )  The  main  task  of  this  working  group  was 
to  identify  and  define  the  physical  parameters 
which  characterise  and  determine  the  fidelity  of_. 
visual  systems,  and  to  recommend  techniques  for 
measuring  these  parameters.  This  paper  is  based 
on  the  findings  of  the  working  group. 

Considerable  progress  has  been  made  in  vi¬ 
sual  simulation  technology,  particularly  in  the 
area  of  computer  image  generation.  It  is  still 
necessary,  however,  to  make  many  compromises  dur¬ 
ing  the  design  of  a  visual  system  before  finding 
a  solution  which  is  both  technically  sound  and 
economically  feasible.  Ideally,  the  tradeoff  de¬ 
cisions  which  must  be  made  require  answers  to  the 
following  questions: 

(a)  What  visua]  cues  does  a  pilot  use 
when  flying  aircraft  in  specific 
missions  or  tasks? 

(b)  How  should  the  characteristics  of  the 
visual  simulation  system  be  specified 
to  enable  those  cues  to  be  presented 
to  the  pilot  with  sufficient  fidelity 
to  allow  the  given  task  to  be  per¬ 
formed? 

^c)  How  well  should  a  pilot  be  able  to 

perform  to  obtain  useful  training,  or 
to  provide  useful  data  to  the  research 
engineer  or  psychologist,  or  to  enable 
his  performance  in  an  aircraft  to  be 
predicted? 

Some  of  these  questions  have  been  addressed 
by  the  ACARD  joint  working  group  of  the  Aerospace 
Medical  and  Flight  Mechanic’s  Panels  on  fidelity 
requirements  of  simulation  for  training  purposes. 
(2) 

It  is  apparent,  however,  that  complete  an- 
rwers  to  these  questions  do  not  exist  and  that  the 
designer  must  rely  on  a  combination  of  data  curr¬ 
ently  available,  experience  with  previous  simula¬ 
tions  and  intuition. 


The  FMP  working  group  10  was  neither  tasked 
with  nor  qualified  for  the  address  of  the  three 
questions,  but  restricted  itself  mainly  to  the 
question  of  which  parameters  should  be  measured 
and  how  they  should  be  measured.  If  we  are  to  be 
able  to  relate  pilot  performance  to  n  set  of  phy¬ 
sical  parameters,  we  must  have  a  relevant  set  of 
parameters  and  reliable  methods  of  measurement. 

The  working  group  divided  the  physical 
parameters  into  three  basic  categories  of  spatial, 
energy  and  temporal  properties  corresponding  to 
the  fundamental  quantities  of  length,  mass  and 
time.  The  following  is  a  discussion  of  certain 
aspects  of  each  category. 

SPATIAL  PROPERTIES 

This  section  of  the  report  covers  metrics 
associated  with  field  of  view,  depth,  geometric 
distortions  and  scene  content.  The  last  of  which 
is  sufficiently  intriguing  to  be  discussed  at 
length.  The  term  scene  content  is  used  here  to 
describe  scene  complexity  or  image  detail  density 
and  is  not  related  to  the  artistic  nature  of  the 
scene.  It  is  recognised  that  artistry  can  have  a 
considerable  impact  or.  the  usefulness  of  a  scene 
but  no  attempt  was  made  to  define  this  parameter 
in  the  report. 

The  current  nature  of  computer-generated 
systems  does  not  allow  highly  detailed  images. 
Users  are  constantly  demanding  more  detail.  There 
is  evidence  to  suggest  that  an  observer’s  ability 
to  judge  his  position  and  velocity  relative  to  his 
immediate  environment  depends  not  only  upon  the 
quality  and  geometry  of  the  visual  stimuli  but 
also  upon  their  quantity.  Answers  to  the  second 
.question  raised  in  the  introduction  therefore  may 
require  measurement  of  image  content.  These  mea¬ 
sures  may  assist  users  to  specify  their  require¬ 
ments  . 

Cohen  et  nl3  measured  the  spectrum  of  the 
luminance  in  natural  scenes  by  analysing  tele¬ 
vision  video  signals.  They  found  that  the  lumi¬ 
nance  spectra  of  natural  scenes  could  be  approx¬ 
imated  by  a  function  that  varied  with  the  inverse 
square  of  the  spatial  frequency.  They  also  found 
that  the  power  spectrum  of  randomly  spaced  lumi¬ 
nance  steps  having  a  random  distribution  of 
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luminance  values  also  exhibits  the  inverse  fre¬ 
quency  squared  function.  They  speculated,  there¬ 
fore,  that  natural  scenes  can  be  thought  of  as  an 
array  of  step  luminance  transitions  having  random 
spatial  and  magnitude  distributions. 

'Hi is  led  one  member  of  the  working  group  to 
speculate  that  scene  content  could  ee  described 
by  the  density  of  the  luminance  or  colour  transi¬ 
tions  within  the  scene. 

The  number  of  luminance  or  colour  transi¬ 
tions  in  a  given  solid  angle  of  a  visual  scene 
may  he  measured  by  an  edge  scanner  that  counts  the 
number  of  observable  edges  in  a  given  scan  direc¬ 
tion.  Take  several  samples  across  the  scene  in 
orthogonal  directions  and  obtain  an  average  number 
of  edge  transitions  in  each  direction.  The  square 
root  of  the  product  of  these  two  numbers  is  a 
measure  of  scene  complexity.  The  luminance/col¬ 
our  transition  density  is  then  obtained  by  divi¬ 
ding  this  number  by  the  solid  angle  subtended  by 
the  scene.  A  number  of  photographs  wore  analysed 
and  values  varying  from  0.11 /deg. 2  for  a  colour 
photograph  of  a  wooded  canyon  to  0.003/deg.2  for 
a  CGI  country  scene  were  obtained. 

A  certain  amount  of  research  would  be  nec¬ 
essary  to  establish  the  value  of  such  a  metric 
and  whether  other  measures  might  be  more  appropri¬ 
ate. 

ENERGY  PROPERTIES 

Those  properties  associated  with  radiant 
power  were  deemed  to  belong  in  this  section  of 
the  report.  Luminance,  contrast,  colour  and  noise 
were  fairly  easy  to  define  and  established  tech¬ 
niques  exist  for  their  measurement.  Colour  is 
somewhat  interesting  because  little  data  exists 
to  indicate  its  usefulness  in  a  simulator  either 
for  training  or  research.  Resolution,  which 
should  perhaps  belong  in  the  spatial  section,  is 
the  most  difficult  to  specify  and  is  probably  the 
most  familiar  characteristic  of  a  visual  system. 

Our  understanding  of  resolution  is  only 
slightly  greater  than  that  of  scene  content.  The 
report  defined  the  resolution  of  a  visual  system 
as  its  ability  to  present  small  recognisable  de¬ 
tails.  In  recognition  of  the  fact  that  flight 
simulators  invariably  require  moving  imagery,  the 
definition  should  probably  be  modified  to  include 
the  ability  to  present  small  movements  or  changes 
in  shape  of  image  details.  The  ability  of  the 
human  visual  system  to  discriminate  fine  detail 
is  termed  visual  acuity.  The  various  types  of 
visual  acuity  are  defined  as  follows: 

(a)  Minimum  separable  acuity  is  the  abi¬ 
lity  of  the  eye  to  separate  two  small 
obj  ects . 

(b)  Minimum  perceptible  acuity  is  the 
eye’s  ability  to  detect  a  small  ob¬ 
ject. 

(c)  Vernier  acuity. is  the  ability  to  al¬ 
ign  two  objects  such  as  two  straight 
lines . 


(d ^  Stereoscopic  acuity  is  the  ability 
to  detect  a  difference  in  depth 
between  two  objects  using  both  eyes. 

(e)  Motion  acuity  is  the  ability  of  the 
eye  to  detect  the  motion  of  small 
objects  . 

The  significant  aspects  of  each  type  of 
acuity  are  described  in  the  report.  They  are  all 
relevant  to  visual  simulation  system  design  al¬ 
though  usually  only  the  first  of  the  five,  min¬ 
imum  separable  acuity,  is  specified  in  a  procure¬ 
ment.  Minimum  spc.iiiv.ble  acuity  (often  called 
visual  acuity,  or  V.A.)  has  an  important  effect 
on  all  types  of  acuity  but  the  relationship  is 
seldom  linear  and  other  parameters  may  have  even 
greater  significance.  Matsubayashi  (**)  found 
that  reducing  the  acuity  cf  one  eye  to  0.3  had 
little  effect  on  stereoscopic  acuity  whereas 
further  reduction  had  considerable  effect.  Re¬ 
searchers  at  the  Boeing  Aerospace  Company  have 
recently  obtained  similar  results  when  the  acuity 
of  both  eyes  was  reduced. 

Minimum  perceptible  acuity  is  a  function 
of  the  length  of  the  object.  A  human  hair  is 
clearly  visible  on  a  CRT  driven  from  a  television 
camera  even  though  its  width  is  an  order  of  mag¬ 
nitude  less  than  the  size  of  object  which  the 
system  bandwidth  might  be  expected  to  display. 

It  is  interesting  to  note  that  algorithms  emula¬ 
ting  such  performance  are  being  developed  by  CGI 
systems  and  the  results  are  proving  very  effec¬ 
tive  . 

Vernier  acuity  is  an  important  factor  in 
flight  tasks  such  as  hovering,  formation  flying 
and  inflight  refueling.  Continuous  sampling 
processes  such  as  the  horizontal  scan  of  a  normal 
television  camera  allow  vernier  acuity  to  app¬ 
roach  that  of  the  human  visual  system  even  though 
the  minimum  separable  acuity  may  be  quite  low. 
Discrete  sampling  processes  such  as  a  raster 
structure  severely  limit  vernier  acuity  although 
it  is  interesting  to  note  that  relatively  high 
vernier  acuity  can  be  obtained  across  the  raster 
of  a  CRT  providing  the  raster  structure  is  vis¬ 
ually  suppressed.  Temporal  characteristics  such 
as  threshold  and  hysteresis  will  also  affect  both 
vernier  acuity  and  motion  acuity. 

All  the  above-mentioned  types  of  acuity 
are  greatest,  i.e.,  have  lower  angular  thresholds 
in  the  foveal  region  of  the  eye.  Figures  1  and 
2  show  how  V.A.  and  motion  acuity  vary  with  re¬ 
tinal  eccentricity.  The  effect  of  luminance  and 
contrast  on  visual  acuity  are  discussed  at  length 
In  the  working  group  report  but  the  general  ef¬ 
fect  on  V.A.  is  shown  in  Figures  3  and  4. 

Measurement  of  Resolution  (Minimum  Separable). 

The  concept  of  limiting  resolution  has 
been  used  by  the  television  industry  for  many 
years.  It  is,  however,  an  unreliable  measure. 
Differences  of  up  to  25%  can  be  obtained  by  mer¬ 
ely  changing  the  length  and  number  of  lines  in 
the  displayed  bar  patterns  and  even  casual  obser¬ 
vers  can  detect  considerable  differences  in 


524  BEST  AVAILABLE  COPY 


picture  quality  between  systems  having  the  same 
limiting  resolution.  The  most  widely  accepted 
method  for  measuring  resolution  is  the  generation 
of  a  Modulation  Transfer  Function  (MTF)  using  a 
sinusoidally  modulated  test  pattern.  The  input 
test  pattern  is  100%  modulated  and  the  output  mod¬ 
ulation  is  plotted  against  spatial  frequency  which 
produces  a  plot  similar  to  that  shown  in  Figure  5. 
Such  a  curve  gives  a  fairly  adequate  description 
of  the  resolution  characteristics  of  most  displays 
and  can  be  used  to  predict  operator  performance 
for  a. number  of  visual  tasks. 

Care  should  be  taken  when  specifying  or  in¬ 
terpreting  MTF's.  Optical  systems  generally  have 
a  different  MTF  for  radial  and  tangential  lines 
and  any  sampling  techniques  such  as  a  raster  str-  — 
ucture  or  the  shadow  mask  of  a  colour  CRT  will 
have,  a  considerable  effect  on  spatial  frequencies 
greater  than  half  the  sampling  frequency. 

Although  the  MTF  curve  is  a  useful  measure¬ 
ment  for  a  visual  system,  it  does  not  give  a  un¬ 
ique  figure  of  merit  enabling  comparisons  to  be 
made  between  visual  systems.  Charman  and  01in(5) 
proposed  the  use  of  the  area  between  the  MTF 
curve  and  the  threshold  detectability  curve  as  a 
measure  of  image  quality  for  photographic  syst¬ 
ems.  The  metric  was  called  the  threshold  quality 
factor  and  has  been  applied  to  electrooptical 
systems  in  general  under  the  name  of  modulation 
transfer  function  area  (MTFA) .  The  threshold 
detectability  curve  would  normally  be  that  of  a 
human  observer  but  may  be  modified  by  other  sys¬ 
tem  characteristics.  Figure  6  shows  typical  MTF 
curves  for  an  optical  display  system  together 
with  a  threshold  detectability  curve  for  a  typ¬ 
ical  observer.  The  areas  between  the  threshold 
detectability  curve  and  the  MTF  curves  represent 
the  radial  and  tangential  MTFA  for  this  particu¬ 
lar  system  under  the  conditions  of  the  measure¬ 
ment.  Sayder  (6)  has  shown  that  MTFA  can  be  used 
to  predict  probability  of  recognition  of  real- 
world  targets  displayed  on  a  CRT.  Varying  amou¬ 
nts  of  video  noise  were  added  to  elevate  the  de¬ 
tectability  threshold  curve. 

An  interesting  experiment  was  performed  by 
Boynton  and  Boss  (7)  to  determine  the  effect  of 
contrast  and  illumination  on  target  recognition. 

They  presented  arrays  of  small  dark  circles  to  a 
number  of  observers.  Half  the  arrays  had  a 
square  substituted  for  one  of  the  circles  and  the 
task  was  to  say  whether  a  square  was  present  or 
not.  The  illumination  of  the  arrays  and  the  con¬ 
trast  of  the  targets  were  varied  and  the  percen¬ 
tage  of  targets  reported  was  plotted  against  time 
(see  Figures  7  and  8) . 

The  design  of  the  apparatus  was  such  that 
high  contrast  could  be  maintained  at  spatial  fre¬ 
quencies  somewhat  beyond  those  useable  by  the  eye 
and  the  control  of  contrast  was  uniformly  applied 
to  all  spatial  frequencies. 

The  system  resolution  and  observer  thres¬ 
hold  curves  relevan;  to  the  case  of  100%  contrast 
targets  and  variable  luminance  can  be  depicted 
as  in  Figure  9.  The  resulting  MTFA  values  can 
then  be  plotted  against  target  recognition  per¬ 
formance  as  in  Figure  10. 


FIGURE  5  TYPICAL  MTF  CURVE 


FIGURE  6  CONTRAST  MODULATION  CURVE  FOR  A  WIDE- 
FIELD  PUPIL-FORMING  DISPLAY. 
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The  percentage  of  targets  reported  in  three 
seconds  has  been  used  as  a  measure  of  observer 
performance.  A  three-second  search  task  not  only 
seems  relevant  to  many  flight  tasks  but  was  also 
used  by  Boynton  in  his  own  analysis  of  the  data. 
Figure  10  shows  a  surprising  degree  of  correla¬ 
tion  between  MTFA  nd  observer  performance  under 
varying  illumination  levels. 

In  general,  it  would  appear  that  MTFA  can 
be  used  to  predict  observer  performance  in  tar- 
g.t  acquisition  tasks  providing  that  the  system 
MTF  exhibits  normal  behaviour. 

hyiuunic  Aspects  of  Resolution . 

MTF  curves  and  threshold  detectability  cur¬ 
ves  are  normally  obtained  using  static  images. 
I'nfortunately ,  the  image  presented  to  a  pilot  on 
.1  visual  simulation  system  is  seldom  static.  A 
much  more  relevant  metric  would  be  one  that  mea¬ 
sures  dynamic  resolution.  System  MTF  can  be  de¬ 
graded  considerably  by  such  factors  as  TV  camera 
lag,  display  decav  time  and  various  aliasing 
ef  feels. 

Threshold  detectability  curves  will  also  be 
degraded  at  high  image  velocities  by  factors  such 
as  the  image  refresh  rate  and  the  basic  paramet¬ 
ers  of  the  human  visual  system.  It  should  also 
be  realized  that  app’.ving  movement  to  the  exter¬ 
nal  environment  as  depicted  on  a  visual  simula¬ 
tion  system,  rather  than  to  the  aircraft  itself 
as  in  the  re..l  world,  may  effect  the  performance 
of  the  human  visual  svstera. 

Several  techniques  can  be  used  for  the  mea¬ 
surement  of  static  MTFA  but  no  known  techniques 
exist  for  the  measurement  of  dynamic  MTFA.  A 
performance  metric  based  on  tasks  similar  to  that 
used  by  Boynton  and  Boss  seems  to  K  an  attrac¬ 
tive  method  for  measuring  dynamic  resolution. 

TEMPORAL  PROPERTIES 

Establishing  general  procedures  for  tiie 
measurement  of  temporal  properties  is  difficult 
due  to  the  variety  ot  teehr.  iques  used  for  the 
generation  of  visual  scenes.  The  following  char¬ 
acteristics  were  determined  to  he  signit leant  and 
techniques  for  their  measurement  are  described  in 
the  report  of  the  working  group. 

.  Excursion  limits  (i.e.,  maximum  velo¬ 
city) 

lime  lags 

.  Noise 

.  Linearity 
Hysteresis 

.  Thresholds 

These  characteristics  are  applicable  tv  .ill 
types  of  visual  simulation  and  are  particularly 


important  when  considering  pilot  or  aircraft 
performance  in  closed-loop  situations.  ideal lv, 
the  measurements  should  be  made  at  the  pilot 
i-ycpomt « 

Measurements  made  at  the  input  to  the 
display  device  are  usually  more  convenient  al¬ 
though  allowances  should  be  made  for  the  prop¬ 
erties  of  the  display  device  or  video  link  it¬ 
self.  Most  of  the  characteristics  will  be  aff¬ 
ected  in  some  way  or  other  by  the  display,  i.i  ., 
threshold  will  be  affected  by  the  vernier  re¬ 
solution  and  time  lags  will  he  affected  by  the 
refresh  rate  and  response  time  of  the  displays. 

Perhaps  the  most  interesting  asact  of 
the  video  link,  and  one  about  which  we  hav-  very 
little  knowledge,  is  the  illusion  of  smooth  move¬ 
ment  created  by  television  displays. 

The  phenomenon  of  apparent  motion  i rented 
by  a  succession  of  static  images  having  approp¬ 
riate  spatial  a  d  temporal  relationships  lias 
been  known  for  .any  vears.  Apar*  from  its  ob¬ 
vious  use  in  tl:  •  television  and  no:  ion  picture 
industries,  it  -as  provided  an  interesting  ana¬ 
lytical  tool  vi-.ich  has  generated  a  considerable 
amount  of  data  concerning  the  conditions  which 
enable  smooth  continuous  motion,  jerky  motion  or 
succession  to  be  perceived.  Many  of  the  moma- 
1  ies  often  seen  in  current  visual  simulation  sys¬ 
tems  can  he  attributed  to  a  change  in  the  per¬ 
ception  of  apparent  motion  by  the  human  visual 
system.  Stroboscopic  effects  sometimes  see”1 
with  runway  markings  an*  a  good  example  of  this . 
Interlace  crawl  (in  which  one-half  of  the  raster 
lines  seem  to  disappear  while  the  remaining  one- 
half  move  sl.ivly  up  the  screen)  is  .  aused  l  v  the 
interlaced  raster  structure  being  perceived  as  a 
set  of  horizontal  lines  moving  a  single  line 
spacing  in  one  field  time.  The  breakdown  of 
smooth  motion  seen  on  systems  using  random  tex¬ 
ture  may  he  explained  by  Julesz's  ( " )  experi¬ 
ments  with  random  dot  arrays.  He  found  that  tin 
spatial  separat ion  between  successive  exposures 
could  not  exceed  IT  arc  minutes  for  motion  to  he 
seen.  Smooth  notion  can  be  seen  with  discrete 
objects  when  the  spat  ial  separation  is  as  much 
as  4  .  Bradiiiok  (  ')  speculated  that  the  diff¬ 
erence  was  due*  to  two  distinct  levels  of  pro¬ 
cessing  in  the  visual  system  b-'ing  responsible 
for  tin*  interpretation  of  apparent  notion. 

Tin-  notion  is  random  dot  arravs  seems  to 
be  determined  by  a  lower-level  process  based  on 
directionally  selective  neurons  while  t  lie  morv 
general  torn  of  apparent  mot  ion  is  determined 
by  an  upper- l  vc 1  process  using  interpretive 
techniques  b.ise.t  on  the  overall  situation. 

The  question  arises  as  to  whether  or  not 
our  perceptual  exp  rience  of  apparent  notion  is 
different  from  that  of  real  notion  and  whether 
or  not  the  visual  svstem  ch.iracterist  ies  associ¬ 
ated  with  real  not  ion  can  be  applied  to  apparent 
motion.  A  corollarv  to  this  question  would  be 
to  determine  how  the  character  1st  ies  ot"  apparent 
motion  .ire  affected  bv  the  charnel er i :  tics  of  tin- 
displav . 
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CONCH'S  IONS 


The  main  task  of  FMP  Working  Croup  10  was  to 
def in'  the  essential  characteristics  of  visual 
simulation  systems  and  to  recommend  suitable  mea¬ 
suring  techniques.  A  certain  amount  of  research 
is  still  needed  before  a  standard  set  of  proced¬ 
ures  can  be  recommenced  for  all  characteristics, 
however,  such  research  will  have  limited  value 
unless  an  attempt  is  made  l  .>  answer  the  three 
questions  raised  in  the  introduction.  These  que¬ 
stions  can  perhaps  be  summarised  by  one,  i.e., 
what  visual  cueing  is  necessary  in  a  simulator  to 
obtain  effective  training  (or  research)?  One  of 
the  recommendat ions  of  the  joint  ACARU  working 
group  on  fidelity  requirements  for  pilot  training 
was  to  establish  A0.AR1)  working  groups  to  pursue 
the  entire  training  effectiveness  question.  AGAR]) 
cannot  perform  any  research  itself  but  the  mul¬ 
ti  \itional,  multidisciplinary  approach  used  bv 
ACARD  seems  ideally  suited  for  the  task  of  defin¬ 
ing  the  required  research. 

Major  General  Abrahamson  opened  the  1980 
Interservice /Indus try  Training  Conference  by  giv¬ 
ing  the  entire  industry,  including  the  users,  a  G 
rating  as  regards  performance.  An  A  rating  is 
probably  unattainable  as  it  would  imply  perfec¬ 
tion,  however,  it  should  he  possible  te  achieve 
a  B  rating  luring  this  decade  il  a  concentrated 
attempt  is  made  t o  answer  the  training  effective¬ 
ness  question. 

1  AGARP  Advisory  Report  No.  In*,  Charac- 
t ies  of  Flight  Simulator 
Visual  Systems  ( 1 9S 1 ) 

AGARP  Advisory  Report  No..  M‘>,  "idelity 
of  Simulation  for  Pilot  Train¬ 
ing  ( 1 980) 
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ABSTRACT 


"Tn  computer  image  generation  (CIG)  ^spatial  filtering^  refers  to  the  combining  of  tonal  information 
from  scene  features  Inside  and  in  the  vicinity  of  a  pixel  to  form  the  video  for  that  pixel.  Several  inves¬ 
tigators  have  recently  proposed  improved  filters,  validating  their  choices  with  pictures  of  sensitive 
test  scenes.  It  can  readily  be  shown  that  filters  which  produce  the  best  static  scenes  generate  serious 
artifacts  when  applied  to  dynamic  field-rate  update  CIG.  The  investigations  in  the  literature  have  not 
explored  this  topic.  When  some  necessary  conditions  imposed  by  the  temporal  effects  of  interlace- 
scan  systems  are  applied  to  the  algorithms,  the  differences  between  simple  filters  and  the  more  com¬ 
plex  filters  become  quite  mmor,  even  on  the  static  test  scenes.  On  CIG  training  scenes,  designed  to 
simulate  the  real  world,  the  differences  in  results  of  a  variety  of  filters  become  imperceptible.^ 

i 

I 


SPATIAL  FILTERING 

When  computer  image  generation  is  used  for  visual  scene 
simulation,  the  scene  is  computed  as  a  large  number  of 
discrete  values  of  video.  Each  computed  value  applies  to  a 
defined  distance  along  a  scan  line.  This  region,  typically  a 
square  or  nearly  square  area  on  the  view  window,  is  referred  to 
as  a  "pixel 

Video  for  a  pixel  is  for  ced  using  contributions  from  defined 
scene  features  faces,  point  lights,  fog.  texture,  etc  Video  may 
be  formed  from  only  one  point  in  a  pixel,  using  all  information 
inside  the  pixel,  or  using  scene  information  from  a  larger  area 
containing  the  pixel  all  have  been  used  The  contribution  to 
pixel  video  of  scene  features  may  be  uniform.  Gaussian, 
bilinear,  sinei  w  x)/x.  or  any  other  weighting  ..many  have  been- 
used 

The  process  of  forming  pixel  video  according  to  a  prescribed 
set  of  rules  is  generally  referred  »o  as  spatial  filtering.  It  has 
also  been  called  anti-aliasing,  antirastering.  quantization 
smoothing,  and  other  terms.  Just  as  standard  filtering  can  be 
shown  to  be  equivalent  to  time  convolution,  spatial  filtering  is 
equivalent  to  space  domain  convolution  Reference  1  examines 
this  topic  in  some  detail  and  conchuies  that  performing  the 
computations  as  convolution  in  the  space  domain  leaos  to 
more  efficient  implementation  than  computing  in  the  spatial 
frequency  domain  via  transformation  techniques. 

A  number  of  recent1/  reported  studies  (3)  (5)  (6)make  a 

case  for  a  preferred  spatial  filtering  technique.  They  are  il¬ 
lustrated  with  scenes  of  ery  sensitive  test  patterns  which  do 
indeed  show  the  superiority  of  the  technique  being  discussed 
This  ieaves  unanswered  two  important  questions.  Have  the 
temporal  effects  associated  with  interlace  raster  scan  displays 
been  adequately  consider'd?  Are  the  differences  sufficiently 
greai  to  be  of  significance  for  the  scenes  used  in  training 
systems? 

Integration  Filtering 

Figure  t  shows  a  group  of  nine  pixels,  identified  by  their  scar, 
!,.ie  number.  I,  ar.d  pixel  number  along  a  scan  line.  J  Video  is  to 
be  determined  for  the  cen'er  pixel  To  the  knowledge  of  the 
author  no  investigator  has  proposed  that  feature  fragments  out 
Side  a  two  by  two  pixel  regon  surrounding  the  center  make  any 
contribution  to  the  cente*  p  xel  video*  Hence,  only  the  portions 


of  faces  A  and  B  shown  inside  the  dashed  lines  will  be  con¬ 
sidered.  A  weighting  function.  W(I.J)  defines  the  desired  filter 
--the  origin  is  translated  to  the  center  of  the  pixel  being  com¬ 
puted.  For  each  face,  the  color  is  defined  by  the  red.  blue,  and 
green  components  Further,  due  to  a  variety  of  types  of  modula¬ 
tion.  these  will  themselves  be  functions  of  I  and  J;  Ra(I.J). 
Ba(I.J).  Ga(I.J).  Bb(I.J|,  Bb(I.J).  and  Gb(I.J)  Pixel  video  is  com¬ 
puted  by  integrating  over  the  area  covered  by  face  A  the  product 
of  W  and  each  of  the  face  A  color  components,  doing  fhe  same 
for  face  B  (and  others  if  there  are  more  than  two),  and  adding 
the  results. 


Figure  1.  Integration  Filtering 


’Computation  of  combined  op.  ~al  and  electronic  transfer  func¬ 
tion  tor  simulated  displays  c!  electro  optical  viewing  systems 
(forward  looking  infrared  and  'ow  light  level  ’olevisiom  involves 
an  exception  to  this  statement  Weighted  contributions  of  com 
puted  video  horn  arrays  as  large  as  9  «  9  pixels  are  used  to 
determine  the  displayed  video  for  the  pixel  m  the  center 
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Oversampling  Filtering 
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WEIGHT 


In  oversamphng  each  pixel  is  considered  as  an  array  of  n  by  n 
subpixels,  as  i  luslrated  in  Figure  2  for  n  =  4.  The  weighting 
function  is  integrated  over  the  area  of  each  subpixel  and  the 
results  are  stored  in  a  weignting  table  (referred'to  as  an  image 
plane  convolution  table  in  reference  1.  and  as  a  pre-computed 
lookup  table  i; .  reference  2.)  These  weights  are  multiplied  by 
the  face  color  components  at  the  centers  of  the  subpixels,  and 
the  results  are  summed  to  obtain  the  pixel  video. 


Figuie  2.  Oversampling  Filtering 


This  process  is  sometimes  discussed  m  terms  implying  it  is 
an  entirely  different  entity  than  integration  smoothing  as 
discussed  above  It  seens  more  enlightening  to  discuss  it  as  a 
technique  tor  approximating  the  exact  results  of  integration 
smoothing  Thinking  along  these  lines  led  to  the  following 
statement  trom  Reference  3:  "Since  no  improvement  in  image 
Quality  was  perceivable  for  n  greater  than  8.  we  used  this  value 
as  representative  of  exact  area  weighting  '  That  is  fully  in 
agreement  with  our  experience,  applying  to  scenes  using  over 
sampling  made  in  our  laboratory  since  1974  In  fact,  an  n  of  4 
gives  results  whose  differences  from  exact  computation  is 
barely  discernible  on  sensitive  test  patterns  Considering  the 
suable  reduction  m  computation  required  by  the  lower  value  of 
n  it  seems  a  reasonable  choice  tor  real  time  hardware  A  com 
panson  of  results  m  Figures  1  and  2  is  instructive  If  we  assume 
uniform  weigh. mg.  then  integration  smoothing  gives  0  2158A  • 
0  7842B  for  the  pixel,  while  oversampling  smoothing  gives 
0  2188A  0  7813B 


A  Necessary  Condition 

In  preliminary  discussion  to  '  iarifv  some  of  the  concepts 
only  one  dimension  will  be  considered  it  will  be  assumed  there 
is  no  variation  along  scan  lines,  but  oniv  vertical  variation 
Fujure  3  identifies  three  scan  lines,  iheir  subiines.  and  weights 
of  sublme  Conti ibutions  in  formation  of  video  to,  scan  line  •  87 
First  consider  a  uniform  two  lint-  weighting  a  ~b  =  c  -d  If  the 
entire  view  window  is  covered  with  a  smgle  face  of  intensity 
■  1 ",  the  value  of  video  fof  Ine  87  must  be  1  The  value  computed 
from  die  designated  weights  is  2a  *  2b  •  2c  •  2d  This  sum 
must  be  1  so  each  weight  must  be  1  8 

Now  assume  a  horizontal  stripe  of  intensity  t  jus*  covering 
sub*  ne  2  of  line  87  ft  makes  a  contribution  of  1  8  to  line  37  and 
a  contribution  of  1  8  to  line  86.  or  a  total  eonti ibut  on  or  t  4  to 
the  view  window  If  we  shift  this  narrow  strip  vertically  so  that  it 
covers  any  other  sublme  we  find  the  contribution  to  the  window 
remains  uni  hanged  This  pattern  of  weights  thus  meets  a 
necessary  condition  st*  ted  by  AC  Erdahi  as  quoted  m 
Reference  4 
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Figure  3.  Subline  weighting 
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In  order  for  a  set  of  weights,  a.b.c.d  to  meet  Erdahls  condi 
lion  it  is  necessary  that  a  -►  d  -  b  +  c  -  0  25  To  illustrate  that 
♦his  condition  is  far  trom  universally  recognized,  one  in 
vestigator  produced  scenes  for  evaluation  using  •  Nonumform 
weighting  over  a  region  covering  only  one  sampling  period 
Since  only  one  sampling  period  was  covered,  d  =  c  -  0  Sit.^e 
nonumform  weighting  was  specified,  a  =  b  Hence  the 
necessary  condition  could  not  be  met.  and  ‘t  could  be  expected 
without  the  need  for  evaluation  that  the  results  would  he  un¬ 
satisfactory 

Reference  5  proposes  triangular  weighting  of  display  spot  m 
tensity  as  optimum,  m  an  investigation  of  display  of  sampled  in 
formation  This  'epresents  a  reasonable  candidate  tor  spatial 
filter  investigation  Figure  5  shows  the  weights  ••  they  meet  the 
Frdahl  requirement 


Reference  3  works  from  the  ideal  frequency  domain  filter  and 
arrives  at  Wil  Ji  •  Sm  .rli  Sm  t»J)  IJ  as  the  optimum  weighting 
Figure  6  shoe  „■»  *uis  m  one  dimension  scaled  so  the  total  area  is 
1  Here  a  •  d  =  0  2385  b  «  c  -  0  2645  This  filter  does  not  meet 
the  t  nqhfness  invariance  requirement  Nevertheless  as  shown 


The  total  ene.qy  contributed  to  all  display  pixels  by  a 
scene  fragment  should  remain  constant  and  be  mdepen 
dent  of  its  position  relative  to  tr.e  pixel  structure 
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by  the  scenes  reproduced  in  the  paper,  it  gives  excellent 
results.  Thiu  indicates  that  some  departure  from  the  equality 
ideal  is  tolerable  One  might  consider  a  little  "fudging"  of  the 
weights  of  Hgure  6  to  make  a  +  d  =  b+c  One  !  .is  a  strong  suspi¬ 
cion  such  a  process  would  end  up  giving  the  weights  of  Figure 
5. 

One  other  weighting  pattern  will  figure  in  subsequent  discus¬ 
sion,  and  hence  will  be  defined  here.  It  is  the  single-line  uniform 
weighting  illustrated  in  Figure  7.  Applied  in  two  dimensions, 
this  gives  uniform  weighting  over  a  single  pixel.  This  has  been 
the  weighting  function  most  widely  applied  to  real  time  systems 
to  date. 

TEMPORAL  EFFECTS 

Raster-scan  display  devices  paint  first  the  odd-numbered 
scan  lines,  then  the  even-numbered  scan  lines  --  the  odd  field 
plus  the  even  field  comprise  a  frame.  With  U.S.  television  stan¬ 
dards,  each  field  is  1/60  second;  a  frame  is  1/30  second. 

Early  CIG  systems  employed  frame-rate  update.  Thirty  times 
a  second  updated  viewer  position  and  attitude  data  and  moving 
target  location  and  attitude  data,  were  applied  to  generation  of 
video  for  the  next  frame  This  led  to  a  number  of  undesirable  ef¬ 
fects;  a  step  effect  and  a  "comb"  effect  associated  with  high 
contrast  moving  edges,  doubling  of  lights  and  other  small 


Reference  4  addressed  some  of  the  topics  discussed  above, 
but  did  not  explore  the  full  impact  on  the  selection  of  filter 
algoiithms. 

To  extend  Erdahl's  statement  to  deal  with  interlace  effects,  it 
must  apply  not  to  the  frame,  but  to  each  field  generated  Let's 
investigate  quantitatively  what  this  implies. 


Figure  8  shows  a  face  covering  scan  line  87  ai  the  time  the 
odd  field  is  being  calculated.  Lines  86  and  88  get  zero  contribu¬ 
tion,  since  they  are  in  the  even  field.  Lines  85  and  89  get  zero 
since  they  are  outside  the  range  influenced  by  the  face  in  the 
location  shown.  Line  87  gets  a  contribution  of  2a  +  2b  Figure  9 
shows  the  same  configuration  with  the  weights  applicable  dur¬ 
ing  the  generation  of  ihe  even  field.  The  totai  contribution  of  the 
face  to  the  scene  is  2c  +  2d 


figures  10  and  11  show  contributions  when  the  face  is 
"isected  by  a  scan  line  boundary.  In  both  cases  the  total  con¬ 
tribution  is  a  +  b  +  ctd.  The  requirements  established  earlier: 
;-;-t  d-D-t  c=0.25  still  apply  since  even  with  field  ra-e  update 
the  scene  may  be  stationary 
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Figum  5.  Triongulor  Weighting 


Figure  6.  Sin  ( ir 0 
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Figure  7.  Single  Line  Uniform 
Weighting 


features  at  certain  rates  of  movement,  and  in  general  a  lack  of 
smoothness  in  perceived  motion  Updating  the  scene  at  field 
rate  requires  only  modest  increases  m  hardware,  it  solves  or 
greatly  improves  the  effects  listed  above,  but  it  mtroouces  a 
new  undesirable  effect 

Assume  the  filter  of  Figure  7  is  being  used  Assume  a  horizon¬ 
tal  face  one  scan  line  high  on  ihe  view  window  Assume  funner 
that  this  face  is  moving  vertically  at  the  rate  of  one  scan  lm  *  per 
field  time  If  the  face  is  located  on  an  even  scan  line  when  too 
odd  field  is  being  computed,  and  on  an  odd  scan  line  when  the 
even  field  is  being  computed,  it  will  contribute  zero  brightness 
to  the  scene  It  the  converse  is  true  if  it  is  located  on  a  scan  line 
of  the  field  being  generated,  it  will  contribute  its  full  intensity 
twice  per  frame  Both  effects  are  incorrect  If  the  face  »s  moving 
at  a  slightly  different  rate  it  will  appear  and  disappear  If  it  is 
inclined  slightly  frem  the  horizontal  and  moving,  face  breakup 
will  appear 


Summarizing  requirements,  we  now  have  2a  ♦  2b  =  0  5,2c 
♦  2 C  =  0  5,j  +  h  +  c  +  d  =  0  5,a  +  d  =  0  25.  b  ♦  c  =  0  25  The 
uniform  weighting  of  Figure  4  obviously  meets  all  these  re 
quirements.  but  the  requirements  as  r.a:ed  do  not  absolutely 
dictate  this  distribution  Consider  a  0  15,  b  =  0  t.  c  =  0  15.  d 
«  0  1  These  satisfy  the  above  equations  However,  a  look  at 
Figure  12  showing  tho  shape  of  the  distrit  tion  giving  this  set 
of  weights  indicates  it  to  be  absurd  When  an  interlace  display 
system  is  used  with  a  CIG  system  with  field  rate  update,  it  is 
necessary  to  apply  the  uniform  distribution  of  Figure  4  in  the 
scan  line  direction  We  applied  it  on  a  real  time  system  m  our 
laboratory  in  1977.  and  it  not  only  solved  the  twinkling  face  and 
light  problem,  but  unexpectedly  produced  significant  improve 
ment  in  the  interlace-caused  step  effect 

The  above  establishes  required  filter  shape  in  the  vertical 
direction,  but  still  leaves  us  with  a  degree  ot  freedom  m  the 
horizontal  direction. 
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Figure  8.  Face  on  Scon 
Line.  Contribution  to 
Odd  Field 


Figure  9.  Face  on  Scon 
Line.  Contribution  to 
Even  Field 


Figure  10.  Face 
Bisected  by  Scan  Line 
Boundary.  Contribution 
to  Odd  Field 


Figure  1 1 .  Face 
Bisected  by  Scan  Line 
Boundary.  Contribution 
to  Even  Field 


W 


Figure  12.  Weighting  W'hich  Meets  Necessary 
Conditions  Established  So  For;  Yet 
is  not  Satisfactory. 


TWO-DIMENSIONAL  WEIGHTING  PATTERNS 

Figure  13  shows  an  isomeinc  representation  o!  a  two 
dimensional  weighting  pattern  m  which  the  weight  is  displayed 
as  height  It  is  the  one  proposed  in  Reference  3 


W  ^0  067738 


sm(vl)  smrirj) 
I  J 


|.|<1  1j|<i 


Figuie  13.  Isometric  Depiction  of  Two- 

D'  nensionol  Weighting  Pattern 
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;n  which  the  scale  factor  is  chosen  to  make  the  total  integrated 
weight  (the  total  volume  under  the  surface  shown!  i-quai  to  orv1 
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Figure  14  shows  an  alternate  depiction  of  the  same  informa¬ 
tion.  with  separate  portions  of  the  figure  showing  the  portion  of 
the  view  window  within  which  scene  features  affect  video  for 
the  center  pixel,  the  weight  as  a  function  of  J  for  I  =  0.  and  the 
weight  as  a  function  of  I  for  J  =  0.  Tne  weighting  functions  for 
the  filters  investigated  in  this  study  will  be  shown  in  this 
manner. 


Consider  the  subpixel  labelled  "e"  in  the  upper  left  quadrant. 
A  face  exactly  filling  this  subpixel  will  contribute  to  the 
brightness  of  pixel  102  of  line  86.  If  we  mentally  move  the 
weighting  table  to  be  centered  on  ttiis  pixel,  we  find  subject 
subpixel  contributes  a  weight  of  "h"  to  pixel  102  of  line  86.  Con¬ 
tinuing  the  analysis,  we  get  the  following  results: 


Figure  14.  Alternate  Depiction  of  Two-Dimensional 
Weighting  Pattern 


Brightness  Consistency  in  Two  Dimensions 

Figure  15  shows  a  group  of  pixels  surrounding  pixel  103  of 
scan  line  87  The  subpixels  whose  scene  content  contributes  to 
the  video  of  the  center  pixel  are  shown,  with  labels  designating 
the  weight  of  their  contributions  Any  weighting  scheme  will  be 
symmetrical  about  the  vertical  and  horizontal  axes;  hence,  if  we 
consider  the  weights  in  the  upper  left  quadrant  the  same  results 
will  apply  to  the  remaim  'g  quadrants 
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Figure  15.  Subpixel  Weight  Identification 
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Integrating  the  defined  weighting  of  Figures  13  and  14  over 
each  subpixel,  we  get  the  following  as  the  upper  left  quadrant 
of  weights: 


0.000919 
0  003021 
0005009 
0  006214 


0.003021 

0.009937 

0.316475 

0.020439 


0.005009 
0016475 
0027316 
0  033888 


0.006214 

0020439 

0.033888 

0041736 


Applying  these  numbers  to  the  results  tabulated  just  above, 
we  find  that  a  feature  just  covering  subpixel  "e"  contributes 
0  0702  to  view  window  brightness.  b|  'Contributes  0.0624.  bj  con 
tributes  0  0624.  and  a"  contributes  0  0551  Obviously  Erdahl  s 
condition  is  not  met  Each  subpixel  should  contribute  0  0625  to 
total  brightness  If  we  envision  an  array  of  snbpixe*  sized 
features  with  one  such  feature  in  each  pixel,  as  the  features 
move  relative  to  the  raster  structure  brightness  varies  by  0  0151 
or  24  2 

The  weighting  used  above  to  illustrate  the  concepts  was 
selected  as  a  candidate  for  evaluation  because  it  has  a  sound 
theoretical  foundation  and  has  given  excellent  i  suits  m 
tests<3>  Designate  it  as  weighting  A  for  subsequent  discus 
sion 

Weighting  "B".  Figure  16  is  as  close  as  you  can  get  to 
weighting  "A",  while  properly  handling  the  temporal  effects 
associated  with  field  rate  update  and  interlace  scan  display 
systems 

w 


0 


Figure  16  Weighting  B.  Horizontal:  Sin  J  J 

Verticol:  Uniform  (“A”  modified  to  meet 
inferloce  temporal  requirement 0 
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Weighting  "C",  Figure  17,  defines  a  pyramid  over  the  region 
of  the  view  window  contributing  to  pixel  video.  It  is  the  two- 
dimensional  equivalent  of  the  weighting  of  Figure  5,  discussed 
in  Refererce  5. 

Weighting  •’D",  Figure  18,  is  weighting  •  J"  modified  to  meet 
interlace  requirements. 

Weighting  "E".  Figure  19,  defines  a  cone  over  the  pixel.  It  is 
one  which  was  proposed  in  Reference  2. 

Weighting  '  F".  Figure  20,  is  the  one  line  by  one  pixel  uniform 
weighting  v.nich  has  been  the  one  most  com.nonly  used  in  real 
time  systems. 

Weighting  "G".  Figure  21.  is  a  uniform  two-line  by  one-pixel 
weighting.  It  was  applied  to  a  real-time  laboratory  system  in 
1977  and  not  only  cured  the  problem  of  narrow  faces  disappear¬ 
ing.  but  also  greatly  improved  other  interlace  arti.acts  such  as 
comb  effect  and  interlace-step  effect. 

Weighting  "H".  Figure  22.  is  two-line  by  two-pixel  uni’orm.  in¬ 
cluded  to  determine  its  effects. 


The  tests  were  made  using  a  laboratory  software  scene 
generation  system,  with  capability  to  imp'ement  any  desired 
spatial  filtering  merely  by  inputting  the  desired  set  of  weights. 
Part  of  the  system  is  a  time-lapse  video  disc  recorder  which 
allows  sequences  to  be  produced  in  slow  time  and  viewed  in 
real  time. 
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Figure  17.  Weighting  C.  Pyramid 
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Figure  19.  Weighting  E.  Cone 
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Figure  20.  Weighting  F.  One  line  by  one  Pixel 
uniform 
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Figure  22.  Weighting  H.  Two  line  by  two  pixel 
uniform 

Limitations  of  Pure  Oversampling 

In  pure  oversampmg.  when  the  center  of  a  subpixel  falls  in¬ 
side  a  face,  that  subpixel's  contributions  to  video  for  pixels  are 
based  on  the  coloi  of  that  face.  If  we  consider  that  an  incremen¬ 
tal  movement  of  an  edge  may  cause  it  to  gam  or  lose  one  sub¬ 
pixel  this  will  cause  an  abrupt  change  in  pixel  video  on  the  order 
of  1/16  to  1/64  o*  the  total  contribution,  depending  on  the 
specific  filtering  algorithm  being  applied  However,  with  certain 
orientations  of  edges  (vertical,  horizontal.  45°)  an  incremental 
change  in  edge  position  can  cause  it  to  cross  the  centers  of  a 
group  of  subpixels,  causing  changes  in  pixel  video  much 
greater  than  indicated  above. 

Figure  23  illustrates  the  type  of  problem  this  can  lead  to.  The 
face  bounded  by  the  dashed  lines  has  a  brightness  of  128  ••  the 
background  has  a  brightness  of  zero  To  really  make  this  worst 
case,  weighting  F.  uniform  one  line  by  one  pixel,  will  be  assum¬ 
ed.  Pixels  1 1  and  12  will  have  brightness  of  96  ••  in  each  of  these 
pixels  the  face  contains  12  subpixels.  Pixels  13.  14.  15.  and  16 
will  have  brightness  of  64.  and  17  and  18  will  have  96  again 
Thus  the  scan  line  will  nave  bright  segments  separated  by  s  dim 
segment 

This  effect  is  quite  apparent  on  high  sensitivity  test  patterns, 
but  is  rarely  noted  on  actual  training  scenes.  The  cure  is  to 
designate  subpixels  belonging  to  a  face  based  not  on  subpixel 
centers,  but  in  such  a  manner  that  the  total  number  of  subpixels 
designated  most  closely  approximates  the  total  pixel  area 
covered  by  the  face  When  properly  done,  this  results  m  a 
system  in  wh.ch  an  incremental  movement  of  an  edge  wili 
result  m  imperceptable  change  in  the  scene 


Selection  of  Test  Scene 

If  one  wishes  to  make  a  valid  comparative  evaluation  of  a  new 
computer,  he  would  be  well  advised  to  apply  it  to  benchmark 
piograms  which  have  been  used  on  prior  computers.  Similar 
standardizasion  would  be  o’  merit  in  spatial  filtering  mvestigr 
tion.  In  the  past,  different  workers  have  applied  their  techniques 
to  a  variety  of  test  patterns.  Of  these,  the  most  sensitive  ap 
pears  to  be  the  one  used  in  Reference  4 

The  pattern  consists  of  triangles  radiating  from  a  common 
center.  The  triangles  arc  defined  as  maximum  intensity 
polygons  against  a  black  background  At  the  periphery  of 
the  pattern  each  triangle  is  one  pixel  wide  and  the  space 
between  neighboring  triangles  is  four  pixels 

Each  of  the  filter  weights  discussed  earlier  was  used  to 
generate  scenes  for  evaluaiion  using  the  described  test  scene 

Scene  Evaluation 

The  following  comments  are  based  on  evaluation  of  8  x  10 
photographs  of  the  test  scene  made  with  various  filter  weights 
It  is  impossible  to  predict  the  degree  to  which  the  subtle  dif 
ferences  will  survive  the  process  of  size  reduction  and  halftone 
printing  ••  which  can  itself  add  further  Moire'  effects  to  the 
scene 

Figure  24  shows  the  test  scene  without  filtering  -  each  pixel 
gets  either  face  brightness  or  background  brightness,  depend 
ng  on  the  location  of  the  pixel  center.  We  'ook  a!  this  and 
chuckle.  It  is  hard  to  believe  that  there  was  a  time  when  all  CIG 
was  based  on  such  processing  Not  only  that,  but  scenes  pro 
duced  in  this  manner  are  st'll  providing  training  with  proven 
transfer. 

At  the  other  extreme,  filters  A.  C.  and  E  give  the  best  results 
for  the  static  test  scene,  although  it  is  known  they  would  be  un¬ 
satisfactory  with  a  dynamic  field  rate  update  system  There  are 
only  very  slight  differences  among  these  three  Contrary  to  ex¬ 
pectations.  and  based  on  the  concensus  of  a  number  of 
observers,  the  conical  filter.  *E.  Figure  27  gives  the  best 
results 

Filter  B  (Figure  28)  can  bo  thought  of  as  Filter  A  modified  to 
be  satisfactory  with  held  rate  update  ui--d  mterluce  smoothing 
The  horizontal  filter  shape  determines  the  character  ol  the  near 
vertical  triangles,  and  the  necessary  uniform  vertical  filter 
rietui  mines  the  chaiacter  of  the  near  horizontal  mangles  The 
near  horizontal  triangles  begin  to  exhibit  aliasing  at  a  greater 
distance  from  the  center  *han  the  ncai  vertical 

Similarlv.  Filter  D  fFigur  29  -  can  bo  considered  as  Filter  C  or 
Filter  E  modified  for  temporal  effects,  and  the  analysis  of  the 
appearance  is  the  same  as  that  of  Filter  0 

Figure  30  shows  Filter  F  the  easily  implemented  1  line 
uniform  weight  filter  that  has  been  the  standard  in  real  time 
anti-aliasing  Figure  31  Filter  G  is  a  undorm  weight  ?  tine  In  t 
pixel  filter  Figure  32  Filter  H  is  uniform.  2  Lnes  by  2  pi»e's 
These  three  exhibit  very  minor  differences  As  expected  the 
near-horizontal  triangles  100k  very  much  like  those  on  the 
earlier  uniform  vertical  weight  filters 
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Figu.e  2?  A  Problem  with  Pure  Ovenomping 


537 


Training  Area  Test  Scenes 


A  scene  in  a  VTOL  landing  area  data  base  was  selected  as 
representative  of  scenes  encountered  nr  training  A  viewpoint 
was  selected  which  placed  a  plant  ,.ith  numerous  pointed 
leaves  involving  edges  at  a  variety  ol  angles  near  the  viewer 
Scenes  were  made  using  Filters  E  and  F  the  two  extremes  m 
quality  (aside  from  the  no  filtering  case)  based  on  the  sensitiv'1 
test  scene  It  was  intended  to  print  both:  however,  after  a 
number  of  observers  were  unable  to  detect  any  differences,  it 
was  concluded  the  would  be  a  waste  ot  papev  The  Filter  E  ver 
sion  is  shown  as  Figure  33 


Figure  26.  Test  Scene  With  Filter  C,  Pyramid  Weighting 


Figure  24.  Test  Scene  Unfiltered 


nice!  Weightii 
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Figure  32.  Test  Scene  with  Filter  H,  2  Line  by 
2  Pixel  Uniform 


CONCLUSION 

There  may  be  some  doubt  whether  the  topic  ol  spatial  filter 
ing  really  calls  for  the  amount  of  investigation  and  evaluation 
being  applied  to  it.  In  Reference  6  Franklin  Crow  states:  "...  im¬ 
ages  made  at  low  resolutions  cem  be  perfectly  adequate  i!  alias¬ 
ing  effects  are  sufficiently  reduced."  Whether  or  not  one  fully 
accepts  this  statement,  there  can  be  no  doubt  that  a  0ystem 
with  effective  spatial  filtering  at  one  resolution  will  be  func¬ 
tionally  equivalent  to  a  system  with  less  effective  filtering  at  a 
higher  resolution.  (Some  of  the  early  non-ieal  time  scene 
generation  used  4000  x  4000  resolution  with  no  spatial  filtering 
and  produced  excellent  results.) 

One  cannot  automatically  state  that  the  most  effective  filter¬ 
ing  known  should  be  used.  It  may  bo  significantly  more  expen¬ 
sive  than  the  next  most  effective.  It  may  exhibit  its  superiority 
on  the  sensitive  test  pattern,  but  show  no  discornable  dif- 
ference  on  training  scenes  In  evaluating  the  options,  factors 
such  as  cost  and  system  requirements  must  be  considered  as 
well  as  effectiveness  of  various  filtering  algorithms. 

In  establishing  the  fact  that  uniform  vertical  weighting  is 
essential  to  eliminate  temporal  effects  of  raster  interlace,  this 
study  removed  one  degree  of  freedom  from  those  available  to 
the  filter  algorithm  developer.  The  evaluation  scenes  establish¬ 
ed  that  filteis  exhibiting  significant  differences  when  their  dif¬ 
ferent  weightings  were  applied  in  two  dimensions  showed  less 
significant  difference  when  the  uniform  vertical  weighting  re 
qui rement  was  imposed. 

The  lack  of  detectable  difference  when  fillers  at  the  extremes 
of  the  quality  range  were  applied  to  typical  training  scenes  in¬ 
dicates  the  importance  of  evaTunting  algorithms  applied  to  the 
typos  of  scenes  to  be  used,  and  balancing  any  dilieroncos  in 
results  against  the  cost  differences. 
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Figure  33.  Training  Scene  Using  Fitter  E 
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AbSTRACT 

This  paper  describes  the  basic  applications  of  texture  in  a  low  cost  visual 

system.  Texturing,  as  applied  in  this  paper,  is  defined  as  the  modulation  of  the 
intense  of  a  color  on  a  surface  This  gives  a  more  natural  appearance  to  the 

surface  at  a  •■datively  low  cost  to  the  system.  The  use  of  texture  to  produce 
realistic  movement,  to  create  spacial  effects,  and  to  give  the  viewer  a  realistic 
feeling  of  perspective  greatly  improves  the  training  capabilities  of  visual 

simulation  systems. k 


HISTORY 

Texturing  plane  surfaces  in  a  visual  system 
is  not  an  entirely  new  -.oncept  Bhnn  and 
Newell  took  the  idea  of  mapping  an  image  onto 
a  smooth  surface  and  extended  it  into  mapping 
synthetically  generated  texture  patterns  and 
highlights  onto  surfaces  1 

General  Electric  used  texture  patterns  to 
simulate  three  dimensional  objects  in  a 
system  called  trie  NASA  Surface  Generator, 
The  output  from  tins  digital  ground  plane 
generator  *as  one  of  th -  first  s>  stems  to  use 
the  concept  of  a  nested  hierarchy  of  texture 
patterns 

Later,  edge  generation  was  added  to  the 
system  to  allow  other  surfaces  to  be 

modeled.  The  current  ?ov.  tost  s.  stems  have 
had  the  ability  to  generate  many  surfaces 
using  edge-face  definitions,  but  are  Turning 
to  texture  as  a  way  b  increase  scene 
Complexity 

INTRODUCTION 

The  use  of  texture  in  a  low  cost  system  not 
only  enhances  the  visual  simulation  of  a 
scene  for  a  viewer  or  trainee.  but  also 

allows  the  modeler  much  greater  flexibility 

in  generating  data  bases  As  texture 
modulate*  the  interior  of  the  surface  it  is 

apphed  to  the  modeler  no  longer  h,*s  to  use 
many  surfaces  to  delineate  specific  features 


Low  cost  visual  systems  that  are  limited  to 
several  thousand  calligraphic  lights  and 
several  hundred  surfaces  have  been 
successfully  used  for  commercial  and  military 
pilot  training 

These  systems  are  eftective  fot  training 
activities  that  invoice  take-off,  landing, 
and  navigation,  but  because  of  the  limited 
number  of  available  surfaces,  they  are  not 
able  to  produce  the  scene  complexity 
necessary  for  advanced  training.  The  problem 
has  bci  i  to  '  nd  a  way  to  extend  the  training 
capability  of  these  systems  by  providing  i 
greater  number  of  training  cues,  and  while 
pres  rving  the  system  •;  low  cost 

The  best  solution  found  for  a  low -cost  system 
was  to  provide  a  *ay  o:‘  ad  fmg  information  to 
the  surfaces  used  to  make  up  a  scene.  A 
change  in  the  intensity  or  the  color  across  a 
surface  provides  a  change  in  position  cue, 
while  the  modeling  and  real  time  co  t  is  only 
one  surface.  This  intensity  change  is 
produced  through  the  additior  of  texture. 
The  modulation  of  the  intensity  of  a  color  on 
a  surface  is  provided  by  tabic  lookup  of  the 
intensity  change  in  a  memory  whose  address  is 
a  function  of  position  on  the  surface 
Figures  1  and  2  provide  contrasting  views  of 
the  same  scene  with  ard  without  texture 

Te>.':jre  has  been  added  to  the  NOVOVIEW  SP3T 
vir.iaf  system.  Working  with  texture  on  this 
system  has  provided  more  information  on  the 
way  models  are  <  nstructed  with  texture,  as 
weli  as  th  way  texture  should  lock  and 
perf>,»tn 
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With  the  introduction  of  texture,  there  have 
been  significant  increases  in  training 
capabilities  of  the  system  Advances  have 

been  made  in  such  areas  as  low-level  flight 
and  over-water  data  bases  that  had  previously 
presented  special  problems  for  low  cost 
systems . 


TEXTURE  BUILDING  BLOCKS 

The  basic  texture  bu.ldmg  blocks  are  the 
texture  patterns,  the  scale  factor  for  each 
pattern,  and  color. 

SP3T  texture  modulates  the  intensity  of  the 
color  of  surfaces  in  the  ground  plane,  or  in 
planes  parallel  to  the  ground  plane. 
Individual  texture  patterns  are  stored  in 
memories  and  addressed  as  a  map  of  points. 
Each  of  these  patterns  can  tie  combined 
together  to  modulate  the  intensity  of  a 
surface. 


The  Scale  Factor 

Data  is  get  .-rated  in  seven.'  different  ways, 
each  targeted  towards  a  particular 
application.  f  ne  surface  description  is 
expanded  to  include  not  only  color 
information,  but  the  combination  of  available 
patterns  to  be  used  to  create  the  overall 
texture  of  the  surface.  The  texture  patterns 
are  mapped  onto  the  surface  according  to  the 
scale  factor  specified  by  the  modeler.  The 
scale  factor  is  used  to  determine  the  area  on 
the  ground  that  will  be  covered  by  one 
occurrence  of  the  texture  map. 

The  scale  factor  is  used  by  the  modeler  in 
three  different  ways: 

1.  It  sets  the  working  lit  it  tides  the  texture 
is  to  he  used  at. 

_.  It  controls  the  repetitiveness  of  the 
texture  patterns. 

i.  It.  controls  the  point  at  which  the  texture 
emeiges  into  Lite  su'-W*. 

For  example,  if  a  scale  factor  of  KX)  feet  is 
specified  for  a  texture  pattern,  then  the 
pattern  will  be  repeated  every  100  feet.  The 
modeler  can  specify  the  scale  factor  for  each 
texture  pattern  stored  in  memory,  and  can 
combine  the  texture  patterns  to  create  other 
patterns.  The  careful  selection  of  the 
scaling  factor  for  each  of  the  available 
texture  patterns  gives  the  modeler  the 
ability  to  select  the  points  at  which  more 
detail  will  emerge.  This  means  the  amount  of 

detail  in  the  scene  will  change  according  to 
the  proximity  of  the  viewer  to  the  scene. 

In  the  previous  example,  the  texture  pattern 
was  given  a  scale  factor  of  100  feet.  This 
repeats  the  pattern  every  100  feet.  However, 
if  this  pattern  is  combined  with  another 
pattern  with  a  scale  factor  of  90  feet  the 
combination  of  the  two  patterns  will  not 
repeat  for  900  feet.  And  if  this  combination 
of  patterns  is  combined  with  a  pattern  wr.os? 


scale  factor  is  800  feet,  the  combination  of 
the  three  patterns  will  not  repeat  for-  7200 
feet.  The  least  common  multiple  of  the  scale 
factors  is  the  point  at  which  the  combination 
of  the  patterns  repeats.  Tins  is  useful  in 
minimizing  the  repetd  iveness  of  texture 
patterns . 


The  Clamp  Value 

The  amplitude  of  each  texture  pattern  is 
modulated  to  provide  control  of  pattern 
en  ergence  according  to  the  size  of  the 
pattern  as  it  varies  throughout  the  image. 
This  function  is  based  on  computations  that 
relate  the  scale  factor  to  the  image  size 
projected  into  model  space.  The  r?  10  of 
this  distance  to  the  scale  factor  is  called 
the  clamp  value  .  This  value  is  calculated 
independently  for  each  texture  pattern.  This 
method  of  antialiasing  has  proven  to  be  more 
useful  in  controlling  texture  than  any  other 
method  n't  use.2 

For  example,  a  surface  that  is  textured  with 
two  texture  patterns,  one  with  a  large  scale 
factor  A  and  the  other  with  a  smaller  scale 
factor  B,  will  have  changes  in  the  overall 
texture  pattern  at  different  proximities  to 
the  surface.  As  this  surface  is  approached, 
the  texture  pattern  with  the  scale  factor  A 
will  be  visible  sooner  than  the  pattern  with 
the  scale  pattern  b.  A  clamp  value  is 
calculated  for  each  texture  pattern's  scale 
factor.  The  clamp  values  are  applied  to  the 
patterns  individually.  1  his  allows  the  final 
pattern  to  change  and  emerge  as  a  function  of 
the  viewing  perspective.  The  bandwidth  of 
the  scene  is  controlled  to  allow  only  the 
information  that  is  free  from  unwanted 
aliasing  to  be  viewed. 


Texture  Patterns 

The  modeler  selects  the  texture  patterns  to 
be  used  in  a  particular  data  base.  Texture 
patterns  that  are  useful  in  one  application 
may  not  be  in  another.  For  example,  the 
texture  map  that  is  used  to  create  a  water 
pattern  is  derived  from  several  sine 
functions,  whereas  grassy  patterns  are  taken 
from  a  fractal  like  function. 

Fractal  techniques  have  been  used  to  create 
many  "natural  !oo!  »ng  '  scenes,  as  their 
application  pi  oduces  an  effective  randomness 
in  the  images.1  The  modeler  should 
select  the  texture  patterns  based  on  the 
specific  application  of  the  data  base 

The  texture  patterns  can  move  in  respect  to 
other  texture  patterns,  as  well  as  to  the 
surface  they  are  modeled  on.  This  motion  is 
restricted  to  the  plane  of  the  surface,  but 
the  effect  of  the  movement  is  appealing. 
Mo\  uig  two  or  more  texture  patterns  with 
ivcpect  to  one  another  produces  a  wave  motion 
useful  for  a  sea  scene  or  in  giass,  rotor 
wash  in  water,  or  Just  trailing  a  vehicle. 
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Another  variable  in  applying  texture  is  the 
use  of  contrast  in  the  texture  pattern. 
Contrast  is  distinction  between  the  dark  and 
light  areas  of  the  pattern.  The  contrast  of 
a  pattern  can  be  controlled  to  give  patterns 
that  range  from  subtle  to  very  obvious.  But 
too  much  contrast  may  result  in  an 
unrealistic  scene  that  could  alias  inspite  of 
the  clamping  function  described  in  the 

previous  section.  Too  little  contrast  can 
produce  a  pattern  that  is  so  subtle  that  it 
becomes  ineffective.  The  over  all  frequency 
spectrum  of  a  pattern  combined  with  contrast 
can  yield  a  pattern  that  is  less  likely  to 
require  antialiasing. 

Color 

The  color  of  a  surface  is  chosen  for  its 
particular  application.  Texture  patterns 

interact  with  the  surface  color,  and  can 
create  greatly  diversified  scenes.  The  same 

texture  pattern  that  is  used  to  give  the 
appearance  of  a  grassy  area  on  a  green 
surface  can  be  used  to  create  a  sand  dure 
effect  on  a  brown  or  tan  surface.  The 
appearance  of  a  scene  can  be  va'iod  by  the 
correct  choice  of  color  and  the  applied 
texture  pattern  . 

Application  of  Texture  Building  Blocks 

With  the  texture  building  block  of  pattern, 
scale  factor,  and  color  the  modeler  can  apply 
texture  patterns  on  surfaces  giving  height, 
motion,  and  closure  cues.  This  gives  the 
modeler  on  a  low  cost  system  more  surfaces  to 
use  in  creating  man-made  structures  or 
non-ground  plane  natural  objects. 

These  building  blocks  have  been  used  to 
generate  data  bases  that  range  from  water  to 
desert  and  grass  to  clouds.  Figure  3  is  an 
example  of  the  application  of  texture  in 
producing  a  sea  scape.  The  waves  are  made  up 
or  two  texture  patterns  at  scales  of  271  feet 
and  1023  feet.  These  values  were  chosen  to 
provide  for  working  altitudes  of  100  to  KXX) 
feet.  At  an  altitude  of  1000  feet,  the  blue 
sea  color  is  broken  uo  to  provide  motion 
cues  As  a  p;lot  descent's  below  this 
altitude,  the  smaller  pattern  emerges, 

providing  closure  cues  for  him. 
The  effect  of  the  emergence  of  the  texture 
can  best  be  demonstrated  on  the  vi<ua!  system 
itself,  or  in  a  motion  picture.  Figures  3, 
4,  and  3  show  the  emergence  of  the  texture  as 
the  position  of  observer  changes  in  three 

static  pictures.  The  ship  is  shown  to 
provide  a  distance  reference. 

The  first  view.  Figure  3,  shows  a  very  larqe 
texture  pattern  from  an  altitude  of  1000 
feet.  Figure  4  shows  the  texture  as  the 
second  pattern  is  beginning  to  become  visible 
at  about  250  feet  figure  5  shows  t  ’  i  c 
texture  patterns  from  a  height  of  50  feet, 
where  the  viewer  can  no  longer  see  the  effect 
of  the  larger  texture  j  attern. 


The  scale  factor  is  used  to  specifically  tune 
the  data  base  for  its  intended  application. 

The  scale  factors  used  for  a  h<gh  altitude 

flight  path  would  be  different  than  those 
used  for  low  level  flight  or  for  maritime 

applications.  The  scale  factor  is 

incorporated  into  the  date  base  during  the 
design  process  and  gives  a  system  the 
flexibility  to  be  used  for  many  design 

applications . 

APPLICATION  OF  TEXTURE 
IN  DATA  BASES 

The  addition  of  texture  to  the  NOVOVIEW  SP3T 
product  line  has  solved  many  of  the 
traditional  problems  that  face  data  base 
designers;  specifically  how  to  incorporate 
the  scene  complexity  needed  for  speed  and 
height  cues,  and  at  the  same  time  provide 

enough  real  world  features  to  make  the  date 

base  usable  and  recognizable.  While  the  use 
of  texture  greatly  simplifies  some  design 
areas,  there  are  several  new  parameters 
associated  with  texture  implementation  that 
require  decisions  to  be  made  early  in  the 

design  process,  and  th3t  may  require  possible 
tuning  throughout  construction.  These 
parameters  include  the  design  or  selection  of 
the  texture  patterns,  the  size  of  each 
pattern,  and  the  height  and  dynamics  oi  each 
selected  pattern. 

As  with  all  other  elements  of  the  data  base, 

the  texture  features  must  be  selected  for  the 
intended  training  requirements.  For  example, 
if  helicopter  take-off  and  landing  training 
is  the  intended  application,  then  a  texture 
pattern  designed  for  use  at  10, 000  feet  would 
be  wasted  on  the  low  altitude  flight 
simulation  of  the  helicopter. 

I  lie  texture  patterns  can  be  selected  only 
after  the  use  of  the  data  base  has  been 
established.  In  tne  NOVOVIEW  SP3  system,  the 
user  can  choose  a  total  of  four  texture 
patterns  from  a  growing  library  constructed 
by  the  supplier  on  an  in-house  computer  and 
frame  buffer  facility.  Th»  process  of 
textureselection  should  include  the  end-user, 
whenever  possible,  to  meet  the  specific  needs 
of  the  data  base  design. 

The  choice  of  the  texture  patterns  must  be 
evaluated  with  the  combination  of  the 
available  scale  factors  and  their  application 
to  the  various  coiors  to  determine  what  is 
most  suited  for  the  particular  training 
needs.  Fortunately,  system  architecture 
readily  allows  for  the  substitution  of 

different  patterns  or  alterations  in  pattern 
size  during  the  construction  process. 

Once  an  initial  set  of  texture  patterns  lias 
been  selected,  the  allocation  of  system 
resources  can  begin.  Using  texture  for 
natural  features  and  reserving  the  edges  -aid 
faces  (and  light  points)  for  cultural  objects 
is  the  standard  approach.  The  classic 
example  of  this  configuration  is  the  ship  in 
Figure  1.  where  all  of  the  system  capacity 


except  the  sky  and  water  surfaces  is 

dedicated  to  the  ship  and  wake. 

Many  applications  of  texture  are  not  as  clear 
cut  as  the  ship  example.  With  •  an  extensive 
use  of  texture  in  sky  and  cloud  features,  a 
pilot  will  still  require  a  frame  of 

reference.  This  could  be  provided  by  cross 
checks  with  airspeed  and  altimeter 
instruments  that  would  take  place  in  the  real 
aircraft.  The  advantage  of  texture  is  that 
the  frequency  of  these  cross  checks  is  much 
closer  to  real  world  flying  conditions.  I  he 
frame  of  reference  can  be  established  by 
having  items  of  know  size  and  orientation 
along  the  flight  path.  When  system  capacity 
allows  for  :t.  reference  features  should  be 
included  in  the  data  base  design. 

Natural  Scene  Elements 

Texture  s  most  obvious  application  is  to 
large  homogenous  areas  such  as  ocean,  desert, 
snow,  and  sky  cloud  scenes.  With  these 
fairly  simple  cases,  there  are  still 
considerations  to  take  into  account.  If  a 
pattern  is  too  small  or  has  too  much 
contrast,  the  repetition  of  the  pattern  can 
become  obvious  and  distracting.  This  can  be 
solved  by  tuning  the  scale  of  the  pattern  or 
choosing  a  less  regular  one,  such  as  shown  in 
Figu res  G  and  7 . 

Attention  must  also  be  given  to  the  texture 
chosen,  so  th.it  it  doesn't  simply  too  many 
cues.  I  lie  real  world  task  of  filing  at  low 
altitudes  over  large  homogenous  areas  like 
water  or  snow  ran  be  very  hazardous  because 
of  disorientation  and  optical  illusions.  If 
the  textures  creates  a  scene  t  hat  is  too 
solid,  negative  training  could  result.  The 
dynamic  capabilities  of  the  texture  can  be 
used  to  avoid  this  problem.  An  effective 
approach  is  to  move  eacli  of  the  texture  maps 
independently.  In  the  case  of  an  u\er  water 
flight,  the  same  wave’  pattern  can  be  loaded 
into  two  of  the  available  texture  maps.  The 
patterns  can  then  lie  moved  in  opposite 
directions  at  the  same  speed.  The  constant 
interaction  between  the  two  patterns  creates 
a  heaving  effect.  This  same  method  can  be 
used  to  produce  blowing  snow  or  sind. 

Sky  and  cloud  decks  are  also  obvious 
applications  of  texture  On  clear  days,  the 
sky  will  normally  have  some  clouds  that  can 
provide  turning  and  pitch  cues  when  the 
ground  is  not  visible.  A  subtle  texture 
pattern  applied  to  the  sky  and  placed  at  an 
altitude  high  enough  to  not  be  penetrated  has 

a  realistic  appearance  and  can  provide 
turning  cues.  An  example  of  this  is  shown  in 
figure  S.  Cloud  layers,  as  shown  in  figure 
9,  require  a  more  definite  pattern.  Some  of 
the  available  texture  patterns  could  bo 
applied  slightly  above  and  below  the*  selected 
cloud  ceiling  or  cloud  top.  I  his  provides 
parallax  cues  that  have  a  three  dimensional 
appearance.  The  ground  texture  may  be 
simultaneously  used  above  the  horizon  with  a 
different  (usually  huger)  scale  factor. 


When  selecting  ground  patterns,  their 
application  to  the  sky  and  clouds  may  be  a 
consideration 

Texture  can  also  be  effectively  used  on  small 
areas.  As  shown  in  Figure  10,  the  texture 
pattern  is  used  to  simulate  a  metal  mesh. 
This  pattern  is  elevated  to  provide  effective 
parallax  cues  when  viewed  with  other  patterns 
in  the  surrounding  area,  such  as  the  sea 
surface  pattern. 

Cultural  Scene  Elements 

The  use  of  texture  on  man-made  or  man  induced 
scene  elements  is  not  as  frequent  as  on 
natural  ones.  When  applied  to  cultural 

objects,  it  tends  to  be  used  on  relatively 
small  areas.  Figure  11  shows  the  use  of  a 

small  scale  fractal  pattern  to  simulate  the 
surface  and  expansion  cracks  on  a  concrete 
runway  This  is  an  effective  speed  cue 
during  landing  and  taxi  training.  Although 

this  is  a  useful  application  for  texture, 
these  patterns  are  limited  to  the  small  areas 
they  were  designed  for.  Also,  such  small 
areas  are  more  likely  to  alias  when  close  to 
eyepoint.  The  interaction  of  the  pattern 

size  and  the  clamping  function  is  used  to 
maintain  a  solid  scene. 

Because  of  tins,  in  an  application  where 
several  different  small  patterns  are  required 
in  different  parts  of  the  data  base,  a 

management  technique  may  be  used  that  brings 
in  a  new  pattern  as  each  area  is  approached. 
This  allows  the  combination  of  several 
patterns  to  be  applied  to  the  overall  scene, 
including  the  sky,  while  applving  only  one 

pattern  to  small  dedicated  features. 

SUMMARY 

The  overall  implementation  of  texture  'n  the 
NOVOv'lLW  SF3T  has  allowed  experimentation  in 
areas  not  previously  associated  with  low  cost 
systems.  The  use  in  scene  elements  such  as 
smoke,  dust  clouds,  and  ship  wakes  are 

examples  of  the  successful  application  of 
texture.  The  application  of  moving  texture 
on  static  surfaces,  or  static  texture  on 
moving  surfaces,  and  texture  moving 
simultaneously  provide  a  wide  variety  of 
effects.  Different  texture  patterns,  static 
or  dynamic,  when  applied  to  adjacent  surfaces 
of  the  same  color,  produce  results  similar  to 
the  shading  effects  that  are  associated  with 
more  expensive  systems. 

The  addition  of  texture  generation  capability 

to  a  low  cost  system  greatly  increases  *  ho 
application  ran  go  of  that  system.  Not  only 

does  texture  enhance  the  simulation 

properties  of  the  system,  by  producing  images 
that  more  closely  resemble  terrain,  skv  and 
cultural  objects,  but  these  enhancements  are 
at  a  relatively'  low  cost  to  system  overhoacf 
With  the  antialiasing  properties  available, 
texture  allows  the  proper  emorgeiu  e  of 
detail,  making  the  closure  to  objects  more 
realistic . 
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Originally,  texture  was  conceived  as  a  way  of 
breaki  -.g  up  surfaces  to  provide  the  viewer 
with  more  information  about  surface  contours, 
but  the  texture  generating  capabilities 
achieved  through  experimentation  anJ  problem 
solving  have  proven  texture  to  be  a  much  more 
valuable  tool . 

The  application  of  texure,  within  the 
structure  of  a  low  cost  system,  provides  the 
end  user  with  a  data  base  that  simulates  more 
natural  appearing  scenes  and  provides 
extensive  training  cues  not  previously 
available  in  such  a  system. 
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ire  2  -  Ship  in  Texturec 
at  Altitude  of  1000  Feet 


Figure  6  -  Conrow  Airport  with 
Grassy  Texture 


Figure  4  -  Ship  in  Textured  Sea 
at  Altitude  of  250  Feet 


Figure  7  Desert  Scene  with  Tanks 


Figure  5  -  Ship  in  Textured  Sea 
at  Altitude  of  about  50  Feet 
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ABSTRACT 


feasibility  has  been  demonstrated  for  a  hybrid  simulation  approach  which  merges  two  tecnnolo- 
gies,  Computer  Generated  Imagery  (CGI)  and  Computer  Synthesized  Imagery  (CSI),  to  form  Com¬ 
puter  Generated  Synthesized  Imagery  (CGSI).  This  approach  holds  promise  as  a  cost-effective,  at¬ 
tainable  method  of  providing  real-time,  high  detail  imagery  for  visual  and/or  other  sensors,  such 
as  FLIR.  A  videotape  for  a  nap-of-the-earth  flight  was  generated,  demonstrating  the  fidelity  and 
mobility  that  can  be  achieved  using  this  CGSI  hybrid  approach  A  description  of  the  approach 
will  be  provided  along  with  selected  video  frames.  Critical  features  which  were  demonstrated  in¬ 
clude:  merging  of  the  two  technologies,  vertical  movement  for  target  acquisition  and  landing, 
horizontal  movement  towards  and  away  from  objects,  dynamic  occulting  of  3-D  objects,  dynamic 
smoke  and  dust,  and  color  visual  and  IR  imagery^ 


INTRODUCTION 

The  advanced  development  and  increased  use  of 
such  sophisticated  sensors  as:  thermal  imaging:  low 
light,  level  TV  (LLLTV\  and  imaging  radar  have  sig¬ 
nificantly  expanded  the  capabilities  of  military  plat 
forms.  Sensor  technology  developments  have  con¬ 
tributed  both  to  safer  and  more  effective  operations 
and  tactics.  These  gains  in  weapons  system  capabili¬ 
ty,  however,  can  only  be  fuliv  realized  through  effec¬ 
tive  training  programs.  This  training  is  more  difficult 
because,  perceptually,  sensor  imagery,  such  as  in 
fra  red,  is  novel  to  observers  and  therefore  more  dif- 
ficult  to  use  and  interpret  This  problem  is  com 
pounded  under  high  data  rate  workload  conditions 
such  as  low-level  flight.  Training  and  exercising  with 
the  actual  sensor  systems  in  the  real  environment  are 
certainly  valuable  training  and  contribute  to  the  ef¬ 
fective  employment  of  the  sensor  system  in  each  mis 
sion  phase  However,  the  cost  and  availability  of  fuel 
and  other  training  resources,  combined  with  risk, 
have  resulted  in  the  reduction  of  actual  platform 
training  hours  and,  consequently,  the  opportunity  to 
exercise  the  actual  sensor  systems.  In  light  of  this  sit 
uation,  more  attention  must  In*  directed  to  the  de¬ 
velopment  and  use  of  simulators.  One  of  the 
challenges  of  simulation  today  is  the  effective,  coordi 
nated  simulation  of  advanced  sensor  systems  for 


training  high  data  rate  workload  conditions  such  as 
low-altitude  flight  missions,  nap-of-the-earth  <NOE> 
and  shipboard  operation.  In  addition.  .»  highly  de¬ 
sirable  factor  »r.  any  sensor  simulation  approach  is 
the  ability  to  correlate  with  an  out-the-window  visual 
scene. 

To  achieve  the  required  level  of  simulation,  de¬ 
signers  must  consider  a  variety  of  factors  including 
the  aircraft  itself,  its  mission,  the  sensor  system,  and 
the  environment  in  which  the  system  will  he  employ¬ 
ed.  In  addition,  there  are  other  factors  related  solely 
to  the  actual  simulation  of  sensor  operation  which 
must  be  considered. 

Hybrid  system  concepts  such  as  Computer  Generated 
Synthesized  Imagery  <C*(5S1 »  offer  near-term  solution 
for  simulating  sophisticated  sensors  and  correlated 
visual  sensor  imagerv.  Because  of  the  high  potential 
pav-off  from  the  real  time  implementation  of  CGSI. 
NAVTRAEQUIPCEN.  with  support  from  P.V 
TRADE,  has  funded  Honeywell's  Systems  and  Re¬ 
search  Center  to  provide  a  non  real  time  demonstra¬ 
tion  of  CGSI  for  a  NOE  flight  and  a  design  for  real 
time  implementation  of  CGSI.  This  report  sum 
marizes  the  results  of  the  effort. 
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COMPUTER  GENERATED  SYNTHESIZED  IMAGERY 


Weakness  of  CGI 


This  sec* ion  describes  the  merging  of  the  Computer 
Generated  Imagery  (CGI)  and  Computer  Synthesized 
Imagery  (CSI).  CGI  uses  the  computer  to  generate  im¬ 
agery  from  a  data  base.  CSI  uses  the  computer  to  in¬ 
sert  targets/objects  in  real-world  pictures.  CGI 
provides  excellert  control  of  a  scene  to  be  constructed 
and  displayed  for  interaction  in  a  simulation  environ¬ 
ment.  However,  the  fidelity  is  low,  and  as  a  result,  re¬ 
alism  in  the  displayed  scene  ir  poor.  CSI  is  jus,  the 
opposite.  Fidelity  is  high,  but  contr  -i  over  scene  con¬ 
struction  is  restricted.  Figure  1  represents  the  fideli¬ 
ty 'viewpoint  flexibility  of  CGI  and  CSI  and  the  resul¬ 
tant  CGSI  (Computer  Generated  Synthesized  Im¬ 
agery). 


Figure  /.  A  Computer  Generated  Synthesized 
Scene 


Contributions  from  Computer  Generated 
jmagery  (CGI) 

The  strength  of  CGI  is  its  ability  to  generate  surface 
representations.  A  real  or  artificial  surface  can  be 
measured  to  get  elevations  at  specified  points,  usually 
at  intersections  of  a  uniform  grid.  The  surface  can  bo 
constructed  in  a  computer  by  connecting  the  sample 
elevations. 

In  addition  to  realistic  surface  representations,  CGI 
offers  control  over  the  placement  of  objects  on  the  sur¬ 
face.  Since  the  data  of  elevations  is  usually  provided 
with  a  uniform  grid,  the  placement  of  other  objects 
can  be  specified  on  this  same  grid.  Trees,  rocks, 
shrubs,  houses,  and  roads  can  all  have  their  positions 
defined  ir  the  data  bast*  grid  system. 

Correct  illumination  and  perspective  are  the  final  two 
major  contributions  from  CGI.  Correct  illumination  is 
achieved  by  finding  the  surface  normal  for  each  pixel 
displayed.  This  normal  is  used  along  with  line-of- 
sight  and  the  normal  from  the  illumination  source 
plus  amibient  intensity  and  haze  factors  to  compute 
pixel  intensity  values.  Correct  perspective  is  achieved 
because  the  distance  is  a  significant  variable  in  the 
perspective  transformation. 


Although  the  position  of  a.i  object  can  be  accurately 
specified,  correctly  illuminated,  and  displayed  in  cor¬ 
rect  perspective,  the  fine  detail  of  an  objec*  cannot  be 
realistically  represented.  The  current  state  of  the  art 
in  CGI  object  representation  is  such  that  objects  ap¬ 
pear  cartoonish.  Some  scene  elements,  such  as  barren 
terrain,  sand,  and  clouds,  can  be  represented  more  re¬ 
alistically  ihan  highly  detailed  structured  objects  like 
trees  and  grass. 

Contributions  from  Computer  Synthesized 
Imagery  (C$1) 

CSI  uses  photographs  of  real  scenes.  The  objects  in 
the  scenes  are  not  represented  individually;  nor  is  the 
scene  modeled  by  elevation  profiles.  Usually  the 
scene  is  held  static,  while  single  objects,  like  aircraft 
or  tanks,  move  within  the  scene.  The  path  followed  by 
the  moving  objects  is  carefully  laid  out.  In  each 
frame,  object  occlusion  is  resolved  by  decid'ng  which 
objects  are  in  the  foreground  and  which  are  in  the 
background.  This  is  a  manual  procedure. 

The  strength  of  CSI  lies  in  its  use  of  photographs. 
With  currently  available  video  equipment,  the  photo¬ 
graphic  data  can  be  manipulated  in  a  manner 
previously  considered  impossible.  Individual  photo¬ 
graphs  can  be  stored  on  a  video  disc,  along  with  hun 
dreds  of  other  photographs.  Access  is  controlled  by  an 
index  just  as  with  digital  data  stored  on  magnetic 
disc.  Thus,  a  frame  can  be  constructed  by  calling  up 
individual  photographs  and  merging  them  in  a  frame 
buffer. 

Weakness  of  CSI 

The  high-fidelity  CSI  scenes  are  limited  to  the  view 
point  of  the  camera.  That  i ...  one  cannot  drive  through 
a  scene  unless  a  series  of  thi  ough  t he-scene  photo¬ 
graphs  is  used.  For  any  reasonable  size  gaining  area, 
the  number  of  through-the-scene  photographs  become 
prohibitive.  For  limited  applications  with  fixed  ob¬ 
servers,  CSI  may  be  sufficient. 

CGSI:  The  Merge 

CGSI  combines  the  liest  features  of  both  techologies: 
CG!  and  CSI.  A  scene  is  constructed  by  placing  indi 
vidual  high-fidelity  CSI  objects  on  a  specified  CGI 
surface.  A  CGSI  scene  is  constructed  much  like  a  CGI 
scene.  The  surface  elevations  and  object  locations  are 
laid  out  on  a  uniform  errid.  The  individual  objects 
used  in  the  scene  are  transformed  for  perspective  and 
size.  This  includes  size,  position,  rotation,  warp,  and 
intensity  transformations  on  the  image.  The  surface 
may  be*  a  CGI  texture  or  a  series  of  CSI  surface  in¬ 
serts.  The  scene  is  const ructed  bv  placing  the  farthest 
object  first  and  continuing  with  overlays  until  the 
nearest  objects  have  been  pi  acini.  The  CGSI  scene 
may  be  constructed  with  imagery  from  any  portion  of 
the  spectrum— visual,  infrared,  millimeter,  or  radar 
frequencies. 
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HOW  TO  BUILD  A  CGSI  SCENE 

This  section  demonstrates  the  construction  of  a  typi¬ 
cal  CGSI  scene  ’hrough  eight  steps.  After  the  eight 
steps  are  completed,  the  resulting  scene  is  provided  to 
the  video  monitor.  This  process  is  repealed  at  the  60 
Hz  field  rate. 

The  construction  of  a  CGSI  scene  begins  with  the  sur¬ 
faces.  ground,  and/or  sea  and  sky.  The  sequence  con¬ 
tinues  with  the  addition  of  objects,  both  small  and 
large.  The  objects  may  he  trees,  rocks,  hushes,  houses, 
roads,  lights,  vehicles,  ships,  docks,  airplanes,  etc. 
Finally,  tin1  special  effects  are  added  which  include 
smoke,  dust,  clouds,  and  shadows. 

Sky  (Figure  2) 

Sky  is  added  in  segments  over  distant  background. 
By  breaking  the  sky  into  segments,  peaks,  and  val 
leys  form  the  skyline  shown.  In  this  example,  the  sky 
was  broken  into  five  segments.  In  general,  the  lower 
edge  of  the  segment  does  not  not'd  to  ho  straight,  hut 
may  he  curved  or  ragged  to  simulate  rolling  or  sharp 
hills  or  mountains.  The  indiv  dual  segments  art' 
warped  based  upon  minimum  and  maximum  data 
base  elevations  and  upon  viewpoint. 


Figure  ,'i.  Textured  Surface  Scene 
Road  Scene  (Figure  4) 

The  surface  library  may  also  contain  special  surfaces 
such  as  roads,  streams,  and  pords,  In  this  example, 
roads  in  the  data  hast'  library  were  warped  to  fit  the 
screen  coordinates.  This  scene  also  contains  tour  tex 
timed  surfaces. 


Figure  I\  Sky  Scene 


Figure  I.  Fuad  Scene 


Textured  Surface  Scene  (Figure  J3) 

Textured  surfaces  are  also  added  in  segments  to  form 
the  foreground  surface.  The  untouched  region  he 
tween  the  sky  and  foreground  forms  the  distance 
background.  Stored  textured  surfaces  warped  to  lit 
the  screen  coordinates  of  the  surface  polygons,  are 
then  added  to  the  screen  The  intensit  ,  of  the  surfaces 
is  varied  based  uinm  the  range  or  other  parameters 


Small  2D  Objects  (Figure  5) 

Small  t wo  dimensional  '2D'  objects.  les>  'ban  1  16  of 
tile  scene's  area,  are  added  Most  natural  objects  such 
as  trees,  husln  s.  and  rocks  ma\  be  represrn.ed  t ion 
one  side  These  are  called  2D  objects  Objects  winch 
cannot  be  represented  from  one  side  such  as  bouses, 
tanks,  ships,  etc  are  referred  ;o  as  three  Uiinrn>u>nai 
objects  Small  objects  are  processed  b\  less  e\ 


pensive  processing  hard  w*are/soft  ware  than  large  ob¬ 
jects  and  surfaces.  During  the  flight  through  a  scene, 
the  2D  object  will  be  handed  off  to  a  large  2D  process¬ 
or  when  it  occupies  more  than  1/16  of  the  area  of  the 
scene. 


Figure  5.  Small  2D  Objects 


Figure  7.  Multi  Surface  Object  Scene 
Large  2D  Object  Scene  (Figure  8) 

Large  2D  objects  typically  occupy  more  than  1/16  of 
the  scene’s  area.  These  objects  may  be  expanded  so 
that  an  object  will  cover  more  than  the  entire  surface 
of  the  screen. 


Mvlti-View  Object  Scene  (Figure  6) 


The  tank  is  an  excellent  example  of  a  multi-view  ob¬ 
ject.  Multi-views  of  the  tank  are  stored  and  the  cor¬ 
rect  view,  based  upon  the  tank  path,  elevation,  and 
observers  viewpoint,  is  used  in  constructing  the 
seen-*.  The  tank  may  he  moving  and  mav  he  verv 


Figure  H.  Large  21)  Object  Seem 
Special  Effects  Scene  (Figure  9) 


Special  elT-  cts  are  used  for  translucent  surfaces  which 
.i  iie  clouds,  dust,  smoke,  and  shadows.  A  mask 
controls  tin*  transmission  functions  and  a  second  in¬ 
put  word  controls  the  intensity  and  color. 


figure  ti.  Multi  View  (thjeel  Scene 


Multi-Surface  Object  Ccene  (Figure  7 1 

The  house  is  an  excellent  example  of  a  multi-surface 
object  The  house  is  separated  into  three  surfaces,  two 
roof  segments  lone  if  Ix-th  sides  are  identical'  two 
ends,  and  two  sides  The  individual  surfaces  of  the 
house  are  warped  from  a  normalized  view  to  the  pet 
spective  dictated  by  the  screen  coordinates  and  then 
joined  together  Images  of  lh>  type  may  U  con 
structed  for  enemy  objects  if  intelligence  information 
is  presented 


Figure  V.  Special  Effect  h 


Complete  Scene  (Figure  10) 

This  completes  a  CGSI  scene. 


Figure  10.  Complete  Scene 


CGSI  SYSTEM  OVERVIEW 


Figure  11  is  a  functional  overview  of  a  real-time 
CGSI  system. 

Data  Base  Construction 


The  data  base  consists  of  two  very  different  types  of 
data:  the  object  library  and  the  gaming  area. 

The  object  library  contains  images  of  objects  and  sur¬ 
faces.  and  transmissivity  masks  of  special  effects  from 
one  to  many  bands  of  ti.e  spectrum.  This  allows  the 
simulation  of  not  only  the  visual  domain,  but  also  in 
fra  red.  millimeter  and  radar  frequency  sensors.  The 
object  library  may  also  contain  a  mixture  of  21)  and 
.'ll)  images.  Tin  images  may  contain  a  variety  of 
day  night  and  diurnal  conditions.  The  object  library 
consists  of  many  actual  high  fidelity  images  taken  in 
the  real  world  from  sensors  and  stored  in  a  photo 
graphic  manner.  In  constructing  a  high  fidelity  object 
library,  images  from  individual  real  world  elements, 
highly  accurate  models,  artist  drawings,  photographs 
of  enemy  devices,  etc.,  are  restored  (correct  edge  roll 
off*  to  form  "near  perfect"  images.  This  is  achieved  by 
operator  and  machine  controlled  thresholding  (drop 
ping  out  the  background!  intensity  corrections,  re 
alistic  color  control  and  perspective  normalizing. 
Ground  contact  and  height  reference  points  are  also 
added  The  "near  perfect"  objects,  surfaces,  and  spe 
cial  effects  are  stored  on  a  rapid  access  and  high 
speed  data  late  media. 

The  gaming  area  data  base  provides  the  information 
necessary  for  placing  the  contents  of  the  abject  li 
brary.  objects,  surfaces,  and  special  effects  on  a  grri.i 
or  gaming  area.  1  ne  objects  may  In*  placed  by  an  oper 
a  tor  or  in  a  random  manner  by  the  computer  The  ob¬ 


jects  in  the  library  may  be  either  stationary  or 
capable  of  movement. 

The  output  of  thi.s  function  determines  “ What  is  in 
the  scene .  ” 

Vehicle  Simulation  Computations 

The  vehicle  simulation  comp  itations,  based  upon  the 
vehicle  math  model  and  cortrol  inputs,  determines 
the  locations  and  viewing  direction  of  the  visual  or 
sensor  system  for  the  primary  vehicle.  In  addition, 
computation  may  be  performed  on  secondary  vehicles 
based  upon  vehicle  models  and  selected  paths.  The 
output  of  this  determines,  “Where  am  I?" 

Systems  Interface 


The  I  O  of  the  vehicle  simulation  system  and  10  of 
the  CGSI  system  must  interface  in  an  efficient  man¬ 
ner  The  communication  subsystem  is  a  bi-directional 
link  and  buffer  interfacing  the  two  systems.  This 
function  is  the  handshake  and  data  flow  between  the 
systems,  ** Comr  unications.  " 

Field  of  View  (FOV)  and  Coordinate 
Transform  Computations 

The  FOV'  processor  determines  the  presence  of  oh 
jects.  surfaces,  and  special  effects  in  the  scene  under 
construction.  The  output  of  a  transformation  matrix 
<V>  converts  real-world  coordinates  to  screen  coordi¬ 
nates.  This  data  front  the  transformation  matrix  per 
ntits  rapid  testing  and  determines  if  all  or  any  portion 
of  the  objects,  surfaces  and  special  effects  are  present 
in  the  scent*.  To  avoid  testing  for  the  presence  of  ail 
the  objects  in  the  data  base,  a  "smart"  algorithm 
tests  only  those  objects  or  surfaces  which  are  m  the 
proximity  of  the  scene.  The  FOX'  processor  maintains 
a  list  of  objects  in  the  FOX’  and  their  object,  surface  or 
special-effect  channel  assignment.  The  function  of  the 
FOX’  computer  is  to  determine.  "What  can  /  see?" 

Controllers  -  Object/Surface/Special 
Effects 


The  controllers  fan  out  and  process  the  control  func 
lions  generated  during  tue  FOX'  computation  The 
prt»cessed  control  functions  are  passed  to  the  oh 
ject  surface  special  effects  processing  channels.  Tin- 
main  functions  pe)  formed  by  the  controller  are  the 
transform  of  gaming  area  corrdinates  it*  screen  coor 
dinates.  range  from  the  vehicle  to  each  object  in  FOX', 
intensity  of  each  object  based  upon  range,  object  iden 
tification,  and  commands  to  the  object  librar*  base  >r 
the  retrieval  of  the  correct  image  data.  The  function 
of  the  controller  is  to  " Fan  out  FOY  data  and  gener¬ 
ate  precise  control  data.  ” 


Object  Surface  Speci 
Library 


at  Effects 


The  library  stores  the  images  used  to  construct  a 
scene  The  controllers  command  the  selected  images 
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which  are  passed  to  the  piocessing  channel;-.  'I'he 
function  of  *he  library  is  to  "Provide  the  correct  im 
age  upon  command. " 

Object/Surface/Specia!  Effect 
Processing  Channels 

The  individual  processing  channels  pipeline  process¬ 
ors  process  one  object,  surface  or  special -effect  per 
channel.  All  the  processing  channels  operate  in  an 
identical  manner;  large  object,  small  object,  surface  or 
special  effects  Each  processing  chrnnel  modifies  ob¬ 
jects,  surfaces,  or  special  effects  from  the  object  li¬ 
brary  by  the  transformation  specified  by  the  control 
functions.  That  is.  the  object,  surface,  .-.pedal -effects 
processing  channels  change  a  stored  image  (normal 
perspective)  to  scene  conditions  (screen  coordinates) 
by  changing  image,  position,  size,  rotation,  and  warp. 
Image  intensity  is  modified  based  upon  range  and  ob¬ 
ject  type.  The  function  of  the  parallel  pipeline  process¬ 
ing  channels  is  to  “ Modify  each  object ,  surface  and 
special  effect  used  in  the  seen  ",  “ 

Scene  Construction 


The  scene  construction  module  takes  the  individual 
image  from  each  processing  channel,  separates  the 
image  from  the  background,  and  assembles  the  sc  no 
based  upon  range.  Near  objects  occlude  more  distant 
objects  The  high-frequency  edges  generated  by  as¬ 
sembling  a  scene  from  individual  images  are 
smoothed  by  a  (Jaussian  function.  This  operation 
matches  edge  and  internal  frequencies. 

This  function  receives  range  information  from  the 
two  object  controllers.  This  range  is  used  to  determine 
whether  or  not  a  particular  object  is  in  front  of.  or  be¬ 
hind  other  objects  in  the  scene.  If  the  particular  object 
pixel  is  the  closest  occupied  pixel  in  the  scene,  then  it 
will  be  the  pixel  displayed.  This  may  be  termed  a 
"nearest "  algorithm. 

The  scene  .'(instruction  function  accepts  \  ideo  inputs 
from  each  video  channel,  and  from  a  background  level 
source  defined  by  the  KOV  computer  This  function 
outputs  real  time  video  signals  to  the  special  effects 

The  digital  scene  construction  function  contains  tin- 
following  subfunctions;  1 1  object  channel  combination, 
2‘  scene  intensity  adjustment  to  accommodate  scene 
wide  intensity  corrections,  and  *i»  sm.»olhmg  coni 
pensate  for  object  to  object  and  object  to  background 
boundaries  The  function  of  the  scene  construction 
module  is  to  "Assemble  the  picture.  " 

Special  Ettacu 

“Hu*  translucent  special  efTecls  are  added  after  the 
generation  of  the  scene  The  special  effects  module 
adds  the  special  effects  based  upon  rang.*  Special  ef 
feds,  such  as  smoke,  or  dust,  may  occur  ahead  of.  or 
behind  images  in  the  scene  The  intensity  masks 
stored  in  the  object  library  and  processed  in  the  spe 


ci<»l  effects  processing  channel  control  the  trans¬ 
missivity  of  the  special  effects  The  intensity  value  in¬ 
put  controls  the  intensity  color  of  the  special  effects: 
the  black  smoke  and  white  clouds.  I'he  wmtion  of  the 
special  effects  module  is  to  "Add  dynamic  or  static 
translucent  effects. " 


TECHNOLOGY  OVERVIEW 

Figure  12  presents  a  hardware  software  overview.  It 
shows  that: 

1.  Most  of  the  hardware  exists  as  a  complete 
subsystem  or  at  the  card  level. 

2.  The  system  does  not  contain  large  amounts  of  soft¬ 
ware. 

o.  The  system  has  common  components;  the  same 
hardware  and  software  is  used  for  controlling  sur¬ 
faces,  objects,  and  special  effects. 

4.  The  system  is  modular.  The  technology  can  he  con 
figured  for  many  different  applications  by  adding 

subtracting  the  modular  components. 

5.  The  input  images  are  real. 

b.  The  real  time  requirements  of  .’JO  frames  second 
and  100  millisecond  transport  delay  are 
achievable. 

7.  The  key  components  in  the  system  are  th"  oh 
jec  surface  special  effects  processing  channels. 

Qbject/Surface/Special  Effect  (OSSE) 

Processing  ChannefffaHware 

Because  of  the  extreme  importance  of  the  OSSK  proc¬ 
essing  channel  hardware  it  ts  discussed  in  greater 
depth. 

To  obtain  the  correct  intensity,  color,  image,  size,  lo¬ 
cation,  rotation,  and  perspective  the  following  func¬ 
tions  are  performed  on  the  lihrar  data. 

a.  A  high  speed  ‘approximately  100  nsec  samples; 
analog  to  digital  i  cuivertei  comerls  the  object  ull¬ 
age  to  a  digital  formal.  The  digital  format  has  a  12 
pixels  {H*r  line,  active  bn  >525  total'  and 
eight  bits  *H*r  pixel  <25b  gray  shades) 

b.  A  high  speed  memory  card  accepts  the  digital  data 
in  either  the  \  or  Y  axis  The  axis  and  direct  son  of 
loading  is  dependent  oil  the  rotation  of  the  image 
The  data  is  loaded  to  minimize  pixel  compression 
during  the  processing  passes  Hat  her  than  rotate 
an  image  bo  degrees  which  may  result  in  some  itn 
age  loss,  the  data  is  loaded  m  the  perpendicular 
axis  ? at  IK)  degrees'  and  rotated  degrees.  This 
memory  card  also  holds  the  object  image  for  proc¬ 
essing  when  the  optical  disc  controller  is  selecting 
a  new  track  ‘image).  This  card  may  be  omitted  it 
the  objects  are  stored  on  -he  disc  in  degree  m 
creinents  or  if  the  rotations  are  less  than  *  45 
degrees. 
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c.  A  look  up  table  modifies  the  intensity  values  of  im¬ 
ages  (for  range  and  contrast  effects!.  This  opera¬ 
tion  requires  only  a  delay  of  several  pixels. 

d.  A  warp  card  transforms  the  image  in  the  Y  axis  on 
a  line-by-line  basis.  The  starting  point  (offset*  and 
magnification  factors  shift  and  cohipress  or  ex¬ 
pand  the  pixels  of  each  line.  This  operation  delays 
the  flow  of  pixels  by  one  line. 

e.  A  second  identical  high-speed  read/write  X  and  Y 
axis  memory  card  accepts  and  stores  the  trans¬ 
formed  Y  data  for  an  odd  and  even  field  to  form  a 
frame.  After  the  Y  axis  field  is  loaded  in  the  Y 
axis,  the  X  axis  data  is  read  out  by  line,  and  even 
and  odd  fields.  This  buffer  operation  requires  one 
video  frame. 

f.  A  second  warp  card  identical  to  Y  processes  X  axis 
data  by  shifts  and  expands  or  compresses  lines. 
Again,  this  operation  delays  the  image  by  approx¬ 
imately  one  video  line. 

Figure  13  shows  a  possible  hardware  implementation 
of  a  processing  channel. 

CGSI  System  Configuration 

CGSI  uses  a  modular  set  of  building  blocks  which 
may  be  configured  to  meet  specific  training  and  sim¬ 
ulation  requirements.  A  system  may  be  configured  for 
a  wide  FOV  visual  color  display  or  a  narrow  FOV  sen¬ 
sor  display.  In  addition.  CGSI  may  be  added  to  exist¬ 
ing  CGI  systems  as  a  product  improvement  to  im¬ 
prove  realism  and  fidelity. 

CGSI  STATUS 

The  feasibility  of  CGSI  in  non-real  time  has  been 
demonstrated  using  a  video  tape  showing  a  nap-of- 


the-earth  helicopter  flight.  The-  CGSI  demonstration 
video  tape  shown  at  the  conference  was  generated  in 
non-real  lime.  All  the  algorithms,  edges,  perspective, 
dust,  smoke,  shadows,  etc.,  were  tested  in  non-real 
time.  A  real-time  CGSI  system  has  been  designed  for 
N  A  VTRAEQU 1PCEN/PM  TRADE  under  the  contract 
“ Extend  CGSI  Concept.  " 

Because  of  the  high  potential  payoff  from  the  develop¬ 
ment  of  this  hybrid  approach,  the  Navy  and  the  Army- 
have  current  plans  aimed  at  demonstrating  feasibili¬ 
ty  of  this  CGSI  technology  in  real-time.  CGSI  pipeline 
processors  will  be  interfaced  to  NAVTRA 
EQUIPCEN  s  Visual  Technology  Research  Simulator 
(VTRS)  facility  for  evaluating  CGSI  concepts  and  f«r 
demonstrating  combined  multisensor  displays.  Figure 
14  demonstrates  the  high  fidelity  and  realism  of 
CGSI. 
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